Adam
Algorithms AIMech:Adam DRAFT
Aliases: Adaptive moment estimation
Stochastic gradient optimization using bias-corrected exponential estimates of first and second gradient moments to scale parameter updates.
Identity Notes & Concept Scope
Adam specifies adaptive moment-based updates; the local SGD concept does not require these estimates. It is an optimizer, not a training task or a trained model.
Claims & Grounded Evidence
2 Claims
Mechanism
#mechanism
The update divides the bias-corrected first moment by the square root of the bias-corrected second raw moment plus a numerical stabilizer.
Limitation
#convergence-counterexample
In the paper's bounded, projected diagonal optimization setting, the analysis constructs stochastic convex problems on which Adam with fixed moment coefficients beta1 and beta2 in [0, 1), satisfying beta1 < sqrt(beta2), does not converge to the optimum.
⚠️ Curated Limitations & Negative Results
- No universal convergence or benchmark superiority claim is curated.
- AdamW, AdaMax and other variants require their own identity assessment.
- The reported counterexample is an existence result for the analyzed Adam setting, not a statement that Adam fails on every training problem or that all later variants share the result.
- The analysis introduces projection and a decreasing step-size setting, omits debiasing for exposition, and states that its arguments extend to the debiased version; it does not analyze every implementation configuration.
Artificial Intelligence Ontology (AIO) Grounding
Mapped AIO Classes
| AIO Target Class | Mapping Relation | Mapping Rationale |
|---|---|---|
AIO:TrainingStrategy
TrainingStrategy
|
related_match | An optimizer can implement parameter fitting, but the pinned TrainingStrategy definition uses a preprocessing genus; no subclass relationship is asserted. |
Evidence Sources & Primary Literature
2 Sources| Source ID | Publication / Document | Type / Access | Reference & Checksum |
|---|---|---|---|
original-paper
|
Diederik P. Kingma, Jimmy Ba
|
primary paper
full_text
|
ID: arXiv:1412.6980v9
sha256:eab9c73ae2ce…
|
convergence-analysis
|
Sashank J. Reddi, Satyen Kale, Sanjiv Kumar
|
primary paper
full_text
|
ID: arXiv:1904.09237v1
sha256:75a949b0751e…
|
Curation Provenance
Curator: Codex
Method: ai_assisted
Model: GPT-6
Created: 2026-10-11T03:12:53Z
Updated: 2026-10-11T03:33:57Z
Source File:
data/curated/Adam.yaml