Home / Curated / Algorithms / Momentum contrast

Momentum contrast

Algorithms AIMech:MomentumContrast DRAFT

Aliases: MoCo

A contrastive representation-learning mechanism that maintains a queue of encoded negative keys and produces new keys with an encoder updated by an exponential moving average of the query encoder.

Identity Notes & Concept Scope This is original MoCo from the 2020 paper, with its image instance-discrimination realization. The momentum acts on encoder parameters, not on per-example memory-bank embeddings. It is distinct from MoCo v2/v3 and from large-batch end-to-end contrastive training. No normalized identity match was found across all 442 pinned AIO labels/scoped synonyms and 187 prior native IDs/names/aliases. Ignored-inclusive searches of both full checkouts found no discovery-checkout hits and nine unrelated main-environment substrings (Pygments and Sphinx locale/bundle text). Exact neighboring AIO definitions, synonyms and incident edges were inspected; task/network vocabulary does not make these algorithms equivalent.

Claims & Grounded Evidence

1 Claim

Mechanism #method

Initialize key-encoder parameters from the query encoder and maintain a finite FIFO queue of K≥1 encoded keys. For each image draw two augmented views, form normalized query q and current positive key k+, and use loss −log[exp(q·k+/tau)/(exp(q·k+/tau)+sum_(k in queue)exp(q·k/tau))] with tau>0. Stop gradients through the key outputs and queue; update the query encoder from the minibatch loss. In paper Algorithm 1, next update key parameters as theta_k←m theta_k+(1−m)theta_q, m in [0,1), enqueue the keys already computed for this batch, and remove the oldest batch to maintain the queue. The queue decouples dictionary size from current minibatch size; the momentum encoder reduces changes between key representations created at different iterations.

supports Source: Momentum Contrast for Unsupervised Visual Representation Learning · Locator: Sections 3.1–3.3, Equations (1)–(2), Figure 2 and Algorithm 1, printed pp. 9730–9732 / PDF pp. 2–4

⚠️ Curated Limitations & Negative Results

Queued keys were encoded at different parameter states. A slow update aims to improve consistency; it does not make historical keys identical to recomputed current-encoder outputs.
The image realization treats different images as negatives. This is instance discrimination, not verification that all negative pairs belong to different semantic classes.
The paper’s distributed ResNet implementation shuffles the key minibatch across GPUs and restores correspondence after encoding to reduce batch-normalization leakage. Other normalization choices need their own reproduction checks.
The displayed algorithm abstracts queue initialization and requires a valid finite dictionary. Temperature, queue length, momentum and augmentation are tuning choices; no single setting is universally optimal.
This record specifies the original paper’s update order and scope. It does not import later MoCo variants, claim independent samples, or reproduce benchmark training.

Artificial Intelligence Ontology (AIO) Grounding

Mapped AIO Classes

AIO Target Class	Mapping Relation	Mapping Rationale
`AIO:SelfsupervisedLearning` Self-supervised Learning	related_match	This is an algorithm for learning representations from relationships between unlabeled views. The pinned AIO target is a MachineLearningTask; this is a related algorithm-to-task mapping, not equivalence or an is-a assertion.
`AIO:ContrastiveLearning` Contrastive Learning	related_match	The algorithm implements positive/negative representation comparisons. The pinned AIO target is exported beneath DeepNeuralNetwork; this related mapping preserves the algorithm versus network/approach distinction rather than asserting equivalence or subclass membership.

Evidence Sources & Primary Literature

1 Source

Source ID	Publication / Document	Type / Access	Reference & Checksum
`method-source`	Momentum Contrast for Unsupervised Visual Representation Learning Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, Ross Girshick	primary paper full_text	ID: https://openaccess.thecvf.com/content_CVPR_2020/html/He_Momentum_Contrast_for_Unsupervised_Visual_Representation_Learning_CVPR_2020_paper.html sha256:69e11f076c82…

Curation Provenance

Curator: Codex

Method: ai_assisted

Model: GPT-6

Created: 2026-10-11T07:53:16Z

Updated: 2026-10-11T07:54:13Z

Source File: data/curated/MomentumContrast.yaml