AMSGrad
Algorithms AIMech:AMSGrad DRAFT
An Adam variant that normalizes first-moment updates with the coordinatewise historical maximum of second-moment estimates.
Identity Notes & Concept Scope
The historical maximum distinguishes AMSGrad from Adam and from weight-decay decoupling in AdamW. It can be combined with other optimizer choices without becoming identical to them.
Claims & Grounded Evidence
1 Claim
Mechanism
#update-rule
The method keeps first- and second-moment accumulators and a nondecreasing coordinatewise maximum of the latter; its original constrained formulation projects the resulting adaptive step onto the feasible set.
⚠️ Curated Limitations & Negative Results
- The paper's convergence analysis imposes assumptions and schedules; no unconditional convergence or benchmark advantage is claimed here.
- The record describes the original formulation; implementation-specific bias correction and numerical stabilizers require separate comparison.
Artificial Intelligence Ontology (AIO) Grounding
Mapped AIO Classes
| AIO Target Class | Mapping Relation | Mapping Rationale |
|---|---|---|
AIO:TrainingStrategy
TrainingStrategy
|
related_match | An optimizer can implement parameter fitting, but the pinned TrainingStrategy definition uses a preprocessing genus; no equivalence or subclass relationship is asserted. |
Evidence Sources & Primary Literature
1 Source| Source ID | Publication / Document | Type / Access | Reference & Checksum |
|---|---|---|---|
original-paper
|
Sashank J. Reddi, Satyen Kale, Sanjiv Kumar
|
primary paper
full_text
|
ID: arXiv:1904.09237v1
sha256:75a949b0751e…
|
Curation Provenance
Curator: Codex
Method: ai_assisted
Model: GPT-6
Created: 2026-10-11T03:33:00Z
Updated: 2026-10-11T03:33:00Z
Source File:
data/curated/AMSGrad.yaml