Differentiable scheduled sampling
Algorithms AIMech:DifferentiableScheduledSampling DRAFT
A scheduled-sampling training procedure that replaces discrete model-feedback embeddings with differentiable soft mixtures, allowing later sequence losses to propagate credit through earlier relaxed decoding choices.
Identity Notes & Concept Scope
Compared all 442 pinned AIO and 585 native IDs, names, aliases and scoped synonyms; no normalized identity match. Ignored- and hidden-inclusive fixed-string searches of the recovery, older discovery and original Documents checkouts returned 3, 0, 0 lines. All distinct match contents were inspected. Semantic matching covered 85 AIO and 200 native definitions, with unchanged preceding rows verified and new matches read. 4 full annotated AIO node/edge/generated contexts and 3 full native records were inspected. The search hits are pending source leads. The new identity is the named scheduled-sampling training procedure with relaxed feedback and downstream credit assignment. Original ScheduledSampling stops gradients through its discrete choices; GumbelSoftmaxRelaxation is a reusable distributional construction used in one variant, not the full curriculum and sequence-loss procedure. TeacherForcing supplies reference feedback. Relaxed greedy and relaxed sample-based training are source-defined variants within this record; their generic descriptive names are not asserted as globally exact aliases. CurriculumLearning and SoftmaxFunction receive related mappings for schedule context and essential operator, without equating this algorithm with either.
Claims & Grounded Evidence
6 Claims
Mechanism
#relaxed-greedy-feedback
Given preceding-step scores s(y), embeddings e(y) and finite positive sharpening coefficient alpha, form q(y)=softmax(alpha*s)(y) and supply the mixture sum_y q(y)*e(y) to the next recurrent step. With differentiable surrounding computations, later losses have a smooth feedback path through those scores. The partial derivative of the mixture with respect to score s(j), holding embeddings fixed, is alpha*q(j)*(e(j)-sum_y q(y)*e(y)). This is a relaxed decoding input, not a discrete selected token or a change of the reference target itself.
Limitation
#sharp-limit-and-ties
Alpha multiplies scores and acts as an inverse temperature. For a unique maximum, increasing alpha sends the mixture toward that token embedding; the source explicitly assumes no ties in its hard-argmax expression. At tied maxima the limiting mixture averages their embeddings rather than selecting a specified tie-break. Greater sharpness does not guarantee better credit assignment: derivatives may vanish away from decision boundaries and grow near ties. Equal embeddings or insensitive downstream losses can also make the choice path uninformative.
Mechanism
#relaxed-sampled-feedback
The sample-based variant adds independent standard Gumbel noise G(y)=-log(-log U(y)) to scores, with U(y) uniform on(0,1), then uses softmax(alpha*(s+G)) to mix embeddings. Holding the noise fixed makes the relaxed feedback map differentiable, while fresh draws still make training stochastic. At finite alpha this is a continuous surrogate, not a one-hot categorical draw or an automatically unbiased gradient of the original discrete objective. It also differs from simply feeding the deterministic expected embedding under the unperturbed model softmax.
Scope
#reference-and-sharpness-schedules
Training retains reference-target cross-entropy and mixes reference embeddings with relaxed model feedback. The reference-selection probability and alpha are separate hyperparameters that may each vary with training. The reported protocol uses reference sequences for the first epoch, then an inverse-sigmoid reference-probability schedule, while comparing fixed and exponentially annealed alpha. At prediction time the decoder can still output a hard argmax. Relaxation improves the available credit path but does not prove that the original continuation is valid for every changed prefix or that the resulting objective is statistically consistent.
Scope
#constructed-credit-path
In a constructed two-token embedding example with e0=-1, e1=2, scores(theta,0), alpha=2 and later loss(one-half)*(mixed_embedding-1)^2, theta=0.2 yields a nonzero relaxed choice-path gradient of approximately1.147576, agreeing with finite differences. The hard greedy embedding stays at e0 in a neighborhood of that parameter and has zero local choice-path derivative. This calculation isolates the feedback path; it does not claim that all parameter derivatives of a complete hard-decoding recurrent model vanish or that the relaxed gradient always improves training.
Scope
#reported-evaluation-boundaries
The source evaluates specified LSTM systems on IWSLT2014 German-English translation and CoNLL2003 German named entity recognition, with hyperparameters and the random restart selected by validation performance. Best reported relaxed configurations improve the selected test metrics, but not every variant improves its matching scheduled-sampling baseline: annealed sampled NER gives54.20 versus54.60 F1, while fixed sampled translation ties at19.69 BLEU. Always-generated relaxed greedy training works in the reported NER setting but is unstable for translation. These selected-run results and similar reported GPU speeds are not universal accuracy, variance or runtime guarantees.
⚠️ Curated Limitations & Negative Results
- The record covers the source-defined relaxed greedy and relaxed sample-based variants of the named training method; it does not rename the reusable Gumbel-Softmax distribution or every differentiable decoder.
- Finite positive alpha smooths the feedback operator under differentiable surrounding computations. Numerical policies, ties, embedding degeneracy and limiting sharpness can affect gradients.
- The sampled variant remains stochastic. A derivative of its relaxed objective is not automatically an unbiased derivative of the discrete objective, and no universal variance-reduction bound is asserted.
- Reference-target compatibility and the relaxation-to-hard-decoding gap remain potential limitations. Smooth credit assignment is not a proof of a correct generative distribution or successful optimization.
- All six primary pages were read. Constructed score/embedding calculations were executed without fitting; no author code, model training or empirical replication was performed.
Artificial Intelligence Ontology (AIO) Grounding
Mapped AIO Classes
| AIO Target Class | Mapping Relation | Mapping Rationale |
|---|---|---|
AIO:CurriculumLearning
Curriculum Learning
|
related_match | Reference feedback and relaxation sharpness can be scheduled during training. This associates the algorithm with curricula, while the pinned target emphasizes example ordering rather than this feedback procedure. |
AIO:SoftmaxFunction
Softmax Function
|
related_match | Softmax on sharpened scores or Gumbel-perturbed scores is an essential relaxed-feedback operator. The training procedure is neither the mathematical function itself nor a subclass of functions. |
Evidence Sources & Primary Literature
1 Source| Source ID | Publication / Document | Type / Access | Reference & Checksum |
|---|---|---|---|
goyal2017
|
Kartik Goyal, Chris Dyer, Taylor Berg-Kirkpatrick
|
primary paper
full_text
|
ID: https://doi.org/10.18653/v1/P17-2058
sha256:4a88f6d184c9…
|
Curation Provenance
Curator: Codex
Method: ai_assisted
Model: GPT-6
Created: 2026-10-13T07:32:17Z
Updated: 2026-10-13T07:32:17Z
Source File:
data/curated/DifferentiableScheduledSampling.yaml