Mean Teacher
Algorithms AIMech:MeanTeacher DRAFT
A semi-supervised classifier-training procedure that regularizes a student prediction toward the prediction of a same-architecture teacher whose parameters are an exponential moving average of successive student parameters, alongside labeled classification loss.
Identity Notes & Concept Scope
Mean Teacher averages weights, whereas temporal ensembling averages predictions and the Pi-model shares current parameters across two gradient-bearing branches. Sharing an EMA component does not make this labeled classifier-training objective identical to BYOL or DINO, which use different representation targets and objectives without the labeled cross-entropy term. This record describes a training algorithm, not a trained checkpoint or fixed network architecture. The inspected AIO SemiSupervisedLLM class specifically combines LLM pretraining with supervised fine-tuning; its related synonym self-training is not an exact identity for this method. AIO SelfsupervisedLearning excludes human-annotated labels, Distillation describes large-to-small compression, and RegularizationLayer is a layer: none supplies an equivalent class for this algorithm. No normalized identity match was found across all 442 pinned AIO labels/scoped synonyms and 230 preceding native IDs/names/aliases. Full-checkout fixed-string searches included ignored and hidden files in both worktree and main checkout; matches concern prior discovery leads and the Pi-model normalization regression. Ten exact AIO CLASS nodes, their scoped synonyms and outgoing edges, plus five neighboring native records were inspected. The corrected Unicode-aware normalizer preserves Π; the previous Pi-model/Model collision was a lexical defect, not semantic identity (review issue #58).
Claims & Grounded Evidence
1 Claim
Mechanism
#method
Evaluate each input with independently perturbed student and teacher networks of the same architecture. Train the student with cross-entropy on labeled examples and a weighted squared-probability consistency cost between student and teacher predictions on labeled and unlabeled examples. Treat teacher predictions as constant during the gradient update. After obtaining the updated student parameters theta_t, update teacher parameters theta_prime_t←alpha theta_prime_(t−1)+(1−alpha)theta_t. Thus the teacher changes by parameter averaging after training steps, not by backpropagation through its consistency branch. Choose the averaging coefficient and consistency-weight schedule as part of the training setup. The core description does not require a particular initialization or warm-up formula. Unlike temporal ensembling, it does not require a persistent target array indexed by the training set.
⚠️ Curated Limitations & Negative Results
- Teacher initialization, coefficient schedules, loss scaling and network state management remain implementation choices; the record does not assert a universal initialization, alpha value or handling of nonparameter buffers.
- Consistency relies on class-preserving perturbations and can propagate wrong targets; the EMA teacher is not guaranteed to improve every prediction.
- The original experiments include architecture and dataset-specific settings. No benchmark score, universal performance ordering, or large-to-small compression claim is asserted.
- The read supplementary pages cover the convolutional setup and branch gradients, not all later supplement experiments or source code.
Artificial Intelligence Ontology (AIO) Grounding
Mapped AIO Classes
| AIO Target Class | Mapping Relation | Mapping Rationale |
|---|---|---|
AIO:Classification
Classification
|
related_match | This procedure learns class predictions from labeled and unlabeled examples. The pinned AIO target describes a classification task under SupervisedLearning, not this semi-supervised training algorithm; this is an algorithm-to-task association, not equivalence or an is-a assertion. |
AIO:MachineLearningTask
Machine Learning Task
|
related_match | The algorithm specifies how to train a predictor from data. The AIO target is the general learning task; this association does not identify an algorithm with that task or assert an ontology subclass edge. |
Evidence Sources & Primary Literature
2 Sources| Source ID | Publication / Document | Type / Access | Reference & Checksum |
|---|---|---|---|
method-source
|
Antti Tarvainen, Harri Valpola
|
primary paper
full_text
|
sha256:86c2ad510ea2…
|
additional-1
|
Antti Tarvainen, Harri Valpola
|
primary paper
full_text
|
sha256:be823265a22f…
|
Curation Provenance
Curator: Codex
Method: ai_assisted
Model: GPT-6
Created: 2026-10-11T10:12:04Z
Updated: 2026-10-11T10:12:04Z
Source File:
data/curated/MeanTeacher.yaml