Manifold Mixup
Algorithms AIMech:ManifoldMixup DRAFT
A neural-network training procedure that interpolates paired examples’ representations at a selected network layer and interpolates their labels with the same coefficient, then trains the entire network on the mixed representation-target pairs.
Identity Notes & Concept Scope
This is the end-to-end hidden-representation extension of input MixUp, not an alias for MixUp’s input-only case, interpolation of fixed extracted features, or a separate trained model. It uses observed labels, rather than ICT’s mixed teacher predictions on unlabeled inputs. Its name does not guarantee that interpolated representations lie on a true data manifold. No normalized identity match was found across all 442 pinned AIO labels/scoped synonyms and 238 preceding native IDs/names/aliases. Full-checkout fixed-string searches included ignored and hidden files in both worktree and main checkout, returning 34 lines in each from prior discovery, saved reviews and related native records. Ten exact AIO CLASS nodes, their scoped synonyms and outgoing edges, plus five neighboring native mechanisms were inspected. This absence assessment is bounded to the pinned snapshot and inspected semantic neighborhood.
Claims & Grounded Evidence
1 Claim
Mechanism
#method
Select a layer k from an eligible set S, which may include the input layer. Forward a minibatch and its randomly shuffled copy to layer k. Draw lambda from Beta(a,a), a>0, and mix both the paired layer representations and the paired one-hot targets using lambda and 1−lambda. Continue the forward pass from the mixed representation and compute the prediction loss against the mixed soft target. Backpropagate through the whole graph, including the network before the mixing layer, so the representations themselves can adapt. The reported implementation choice uses one layer and one coefficient per minibatch. With only the input layer eligible, S={0}, the objective reduces to original input MixUp; allowing hidden layers is the distinguishing extension. The objective averages over examples, mixing coefficients and eligible layers.
⚠️ Curated Limitations & Negative Results
- Representations being mixed must be compatible at the selected layer. The eligible layers and mixing distribution remain choices.
- The source’s flattening theorem is framed under idealized function-class, dimensionality and minimization conditions; it does not guarantee the same geometry for every finite trained network.
- Interpolated labels impose an inductive bias and need not equal the true conditional targets between examples.
- The paper explicitly distinguishes single-step adversarial results from full adversarial robustness. No empirical robustness or accuracy claim is imported.
Artificial Intelligence Ontology (AIO) Grounding
Mapped AIO Classes
| AIO Target Class | Mapping Relation | Mapping Rationale |
|---|---|---|
AIO:DataAugmentation
Data Augmentation
|
related_match | The procedure creates synthetic representation-target pairs during end-to-end training. This is related to the preprocessing augmentation class but is not merely an input preprocessing transform. |
AIO:SupervisedLearning
Supervised Learning
|
related_match | The method trains against interpolated observed labels; this is an algorithm-to-learning-task association. |
Evidence Sources & Primary Literature
1 Source| Source ID | Publication / Document | Type / Access | Reference & Checksum |
|---|---|---|---|
method-source
|
Vikas Verma, Alex Lamb, Christopher Beckham, Amir Najafi, Ioannis Mitliagkas, David Lopez-Paz, Yoshua Bengio
|
primary paper
full_text
|
sha256:b8ba5ac4b3bd…
|
Curation Provenance
Curator: Codex
Method: ai_assisted
Model: GPT-6
Created: 2026-10-11T10:34:59Z
Updated: 2026-10-11T10:34:59Z
Source File:
data/curated/ManifoldMixup.yaml