Rectified Adam
Algorithms AIMech:RAdam DRAFT
Aliases: RAdam
An Adam-family optimizer that rectifies its adaptive update using a time-dependent estimate of the effective history of the second-moment statistic.
Identity Notes & Concept Scope
RAdam's rectification and early-step branch distinguish it from unmodified Adam and from weight-decay decoupling alone.
Claims & Grounded Evidence
1 Claim
Mechanism
#update-rule
The cited implementation computes a rectification term when its effective-history statistic exceeds five; before that threshold it takes a bias-corrected first-moment step without adaptive second-moment scaling.
⚠️ Curated Limitations & Negative Results
- The implementation exposes coupled and decoupled weight-decay choices; these are distinct from the rectification mechanism.
- The record asserts the update procedure, not a universal variance reduction, convergence result or benchmark advantage.
Artificial Intelligence Ontology (AIO) Grounding
Mapped AIO Classes
| AIO Target Class | Mapping Relation | Mapping Rationale |
|---|---|---|
AIO:TrainingStrategy
TrainingStrategy
|
related_match | This algorithm can implement parameter fitting, but the pinned TrainingStrategy definition uses a preprocessing genus. This is a contextual training relationship, not equivalence or a subclass assertion. |
Evidence Sources & Primary Literature
1 Source| Source ID | Publication / Document | Type / Access | Reference & Checksum |
|---|---|---|---|
pytorch-docs
|
PyTorch contributors
|
official documentation
documentation
|
sha256:a8776d8fee94…
|
Curation Provenance
Curator: Codex
Method: ai_assisted
Model: GPT-6
Created: 2026-10-11T03:25:18Z
Updated: 2026-10-11T03:25:18Z
Source File:
data/curated/RAdam.yaml