Bootstrap Your Own Latent
Algorithms AIMech:BootstrapYourOwnLatent DRAFT
Aliases: BYOL
A self-supervised image representation-learning procedure that trains an online predictor to match a stopped-gradient target projection from another augmented view, while updating the target encoder and projector by exponential moving averages.
Identity Notes & Concept Scope
BYOL is the 2020 online/target learning procedure, not statistical bootstrap resampling or a particular trained network. Only the online branch has the prediction head. Its normalized regression objective has no explicit negative-pair term or historical key queue. No normalized identity match was found across all 442 pinned AIO labels/scoped synonyms and 187 prior native IDs/names/aliases. Ignored-inclusive searches of both full checkouts found no discovery-checkout hits and nine unrelated main-environment substrings (Pygments and Sphinx locale/bundle text). Exact neighboring AIO definitions, synonyms and incident edges were inspected; task/network vocabulary does not make these algorithms equivalent.
Claims & Grounded Evidence
1 Claim
Mechanism
#method
Generate two augmented views v and v′ of each image. The online branch computes p(v)=q_theta(g_theta(f_theta(v))), and the target branch computes z_xi(v′)=g_xi(f_xi(v′)). Normalize both nonzero vectors and minimize their squared distance, plus the reversed-view term p(v′) versus z_xi(v), averaged over the batch. Stop gradients at target projections and optimize only the online encoder, projector and predictor. After the optimizer step update the corresponding target encoder/projector parameters by xi←tau xi+(1−tau)theta with tau in [0,1] and a chosen decay schedule. The target branch has no predictor to update. After training retain the online encoder f_theta and discard the training heads and target branch for the stated downstream representation.
⚠️ Curated Limitations & Negative Results
- The paper’s general same-architecture wording is qualified by its explicit online-only predictor in Figure 2 and the algorithm. Matching encoder/projector weights are averaged; a nonexistent target predictor is not required.
- Absence of an explicit negative-pair term does not prove collapse impossible. The source’s conditional-variance explanation assumes an optimal predictor and simplified unnormalized, unsymmetrized dynamics; it is an intuition, not a general convergence theorem.
- Batch normalization still couples examples. The batch-size ablation also accumulates gradients to control effective update size; it does not establish unconditional batch-size independence.
- Augmentation, normalization, predictor design and target-decay schedule affect training. tau=1 freezes the target and tau=0 copies the online parameters; these endpoints do not by themselves reproduce the usual slow-target regime.
- Vector normalization needs a defined nonzero-norm numerical treatment. No universal transfer benefit, benchmark reproduction or model checkpoint is asserted.
Artificial Intelligence Ontology (AIO) Grounding
Mapped AIO Classes
| AIO Target Class | Mapping Relation | Mapping Rationale |
|---|---|---|
AIO:SelfsupervisedLearning
Self-supervised Learning
|
related_match | This is an algorithm for learning representations from relationships between unlabeled views. The pinned AIO target is a MachineLearningTask; this is a related algorithm-to-task mapping, not equivalence or an is-a assertion. |
Evidence Sources & Primary Literature
2 Sources| Source ID | Publication / Document | Type / Access | Reference & Checksum |
|---|---|---|---|
method-source
|
Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre H. Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Daniel Guo, Mohammad Gheshlaghi Azar, Bilal Piot, Koray Kavukcuoglu, Rémi Munos, Michal Valko
|
primary paper
full_text
|
ID: https://proceedings.neurips.cc/paper_files/paper/2020/hash/f3ada80d5c4ee70142b17b8192b2958e-Abstract.html
sha256:d6a3c810134f…
|
additional-1
|
Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre H. Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhaohan Daniel Guo, Mohammad Gheshlaghi Azar, Bilal Piot, Koray Kavukcuoglu, Rémi Munos, Michal Valko
|
primary paper
full_text
|
ID: https://proceedings.neurips.cc/paper_files/paper/2020/file/f3ada80d5c4ee70142b17b8192b2958e-Supplemental.pdf
sha256:41c79ee8588e…
|
Curation Provenance
Curator: Codex
Method: ai_assisted
Model: GPT-6
Created: 2026-10-11T07:53:16Z
Updated: 2026-10-11T07:54:13Z
Source File:
data/curated/BootstrapYourOwnLatent.yaml