AIMech AI

Temporal ensembling

Algorithms AIMech:TemporalEnsembling DRAFT
A semi-supervised classifier-training procedure that regularizes a current stochastic prediction toward a bias-corrected exponential average of earlier predictions for the same indexed training example, alongside labeled cross-entropy.
Identity Notes & Concept Scope This is the prediction-ensemble algorithm in Laine and Aila, distinguished from the same paper’s Pi-model. Its dataset-indexed stored targets and epoch-based refresh differ from Mean Teacher’s per-step weight averaging. The name does not imply an ensemble of separately trained classifiers at inference. This record describes a training algorithm, not a trained checkpoint or fixed network architecture. The inspected AIO SemiSupervisedLLM class specifically combines LLM pretraining with supervised fine-tuning; its related synonym self-training is not an exact identity for this method. AIO SelfsupervisedLearning excludes human-annotated labels, Distillation describes large-to-small compression, and RegularizationLayer is a layer: none supplies an equivalent class for this algorithm. No normalized identity match was found across all 442 pinned AIO labels/scoped synonyms and 230 preceding native IDs/names/aliases. Full-checkout fixed-string searches included ignored and hidden files in both worktree and main checkout; matches concern prior discovery leads and the Pi-model normalization regression. Ten exact AIO CLASS nodes, their scoped synonyms and outgoing edges, plus five neighboring native records were inspected. The corrected Unicode-aware normalizer preserves Π; the previous Pi-model/Model collision was a lexical defect, not semantic identity (review issue #58).

Claims & Grounded Evidence

1 Claim
Mechanism #method
Maintain zero-initialized N-by-C arrays Z and target z_tilde for N indexed examples and C classes. During epoch t, obtain one stochastic prediction z_i for each example when its minibatch is processed. Minimize labeled cross-entropy divided by the full minibatch size plus w(t)/(C|B|) sum over all i in B of ||z_i−z_tilde_i||², holding the stored targets constant during differentiation. After predictions have been used for that epoch, update each accumulated prediction Z_i←alpha Z_i+(1−alpha)z_i and set the next target to Z_i/(1−alpha^t), with fixed 0≤alpha<1 and t starting at 1. Set the first epoch’s consistency weight to zero and ramp it up. Algorithm 2 displays the target update after the epoch; its caption permits per-example updates within a minibatch after the old target has been used. Predictions gathered within an epoch can come from different current parameter states. The exponential average is over example predictions, not model parameters.

⚠️ Curated Limitations & Negative Results

Artificial Intelligence Ontology (AIO) Grounding

Mapped AIO Classes
AIO Target Class Mapping Relation Mapping Rationale
AIO:Classification
Classification
related_match This procedure learns class predictions from labeled and unlabeled examples. The pinned AIO target describes a classification task under SupervisedLearning, not this semi-supervised training algorithm; this is an algorithm-to-task association, not equivalence or an is-a assertion.
AIO:MachineLearningTask
Machine Learning Task
related_match The algorithm specifies how to train a predictor from data. The AIO target is the general learning task; this association does not identify an algorithm with that task or assert an ontology subclass edge.

Evidence Sources & Primary Literature

2 Sources
Source ID Publication / Document Type / Access Reference & Checksum
method-source
Samuli Laine, Timo Aila
primary paper
full_text
sha256:d19ea2ecc468…
additional-1
Antti Tarvainen, Harri Valpola
primary paper
full_text
sha256:86c2ad510ea2…

Curation Provenance

Curator: Codex
Method: ai_assisted
Model: GPT-6
Created: 2026-10-11T10:12:04Z
Updated: 2026-10-11T10:12:04Z
Source File: data/curated/TemporalEnsembling.yaml