Training signal annealing
Algorithms AIMech:TrainingSignalAnnealing DRAFT
Aliases: TSA
A supervised-loss scheduling procedure that temporarily excludes labeled examples already predicted with confidence above an increasing threshold, gradually allowing more labeled training signal during semi-supervised learning.
Identity Notes & Concept Scope
TSA is the named labeled-loss schedule introduced as an additional UDA training technique, not the entire UDA method, an unlabeled pseudo-label confidence filter, or an LLM model class. It is related to curriculum scheduling but does not follow the pinned AIO CurriculumLearning definition’s simple-to-complex presentation rule: it withholds already-confident labeled examples rather than starting with them. No normalized identity match was found across all 442 pinned AIO labels/scoped synonyms and 243 preceding native IDs/names/aliases. Full-checkout fixed-string searches included ignored and hidden files in both worktree and main checkout, returning 102 lines in the worktree and 103 in main from prior discovery, saved reviews and related native records. Twelve exact AIO CLASS nodes, their scoped synonyms and outgoing edges, plus eight neighboring native mechanisms were inspected. This absence assessment is bounded to the pinned snapshot and inspected semantic neighborhood.
Claims & Grounded Evidence
1 Claim
Mechanism
#method
At training step t, compute the model probability of each labeled example’s known correct class. Keep that example in the supervised loss only when its probability does not exceed eta_t, following the inspected author implementations; hold the mask fixed during differentiation. Sum the retained cross-entropy losses and divide by max(number_retained,1), producing zero when none are retained. Set eta_t=a_t*(1−1/K)+1/K for K classes and total step count T. The source provides linear a_t=t/T, log a_t=1−exp(−5t/T), and exponential a_t=exp(5*(t/T−1)) schedules. The linear schedule spans exactly 1/K to 1 over t=0..T; the exponential begins slightly above 1/K and reaches 1, while the log schedule begins at 1/K and ends slightly below 1. These exact endpoint distinctions follow the published formulas. The threshold suppresses easy labeled examples early; this is separate from the unlabeled UDA confidence filter, which accepts high-confidence targets and averages masked losses over the full unlabeled batch.
⚠️ Curated Limitations & Negative Results
- The schedule depends on the current classifier’s confidence and does not measure intrinsic example difficulty or guarantee avoidance of overfitting.
- The paper’s prose describes a threshold progressing from1/K to1, but the nonlinear formulas do not both attain those exact endpoints at finite T.
- The implementations retain threshold equality and normalize by retained count; these details differ from a strict less-than mask or a full labeled-batch denominator.
- The linear/log/exponential schedule choices and K,T must match the training setting. No benchmark gain, optimal schedule or universal need for TSA is claimed.
Artificial Intelligence Ontology (AIO) Grounding
Mapped AIO Classes
| AIO Target Class | Mapping Relation | Mapping Rationale |
|---|---|---|
AIO:CurriculumLearning
Curriculum Learning
|
related_match | Both concern scheduling training examples, but TSA filters already-confident labeled samples instead of the target’s simple-to-complex presentation order. The relation is related, not broader or equivalent. |
AIO:TrainingStrategy
TrainingStrategy
|
related_match | TSA schedules a loss contribution during training; the target is a broadly worded preprocessing/training-strategy class at another abstraction level. |
Evidence Sources & Primary Literature
3 Sources| Source ID | Publication / Document | Type / Access | Reference & Checksum |
|---|---|---|---|
method-source
|
Qizhe Xie, Zihang Dai, Eduard Hovy, Minh-Thang Luong, Quoc V. Le
|
primary paper
full_text
|
sha256:77533ff314e4…
|
image-code
|
Google UDA Team Authors
|
official documentation
documentation
|
sha256:1d4f91ee5df3…
|
text-code
|
Google UDA Team Authors
|
official documentation
documentation
|
sha256:f34e4a9d3654…
|
Curation Provenance
Curator: Codex
Method: ai_assisted
Model: GPT-6
Created: 2026-10-11T10:45:16Z
Updated: 2026-10-11T10:45:16Z
Source File:
data/curated/TrainingSignalAnnealing.yaml