Unsupervised Data Augmentation

Compute supervised cross-entropy on labeled examples. On an unlabeled input, obtain the current model’s probability vector from the original or weakly augmented view and hold that target fixed during differentiation. Train the model on a stronger augmented view to match this soft distribution, adding a weighted mean consistency loss over the unlabeled batch to the labeled loss. Main-paper Equation1 uses cross-entropy; the inspected image and text implementations instead compute forward KL divergence from the fixed target to the augmented prediction. These objectives differ by target entropy in value but have the same augmented-prediction gradients when targets and masks are held fixed. The additional confidence filter retains an unlabeled example only when the maximum unsharpened target probability is strictly greater than beta. Target sharpening uses softmax of logits divided by a positive temperature. The masked consistency losses are averaged over the entire unlabeled batch, not the retained subset. The source uses image transformations, back-translation or TF-IDF word replacement depending on the task. Its particular image RandAugment implementation samples two operations from 15 types, each with its own sampled magnitude in [1,10) and application probability 0.5; it is explicitly a variant of the separately represented shared-magnitude RandAugment. Training-signal annealing can additionally filter easy labeled examples, and a separate domain-relevance procedure can select external unlabeled data; neither is silently folded into the core consistency equation.