Virtual adversarial training

For a fixed parameter snapshot theta_hat, approximately maximize D(p_theta_hat(y|x)||p_theta_hat(y|x+r)) over ||r||_2<=epsilon; the reported classifier implementation uses KL divergence. At r=0 the smooth KL objective has zero value and first derivative. Its local quadratic approximation gives a dominant-Hessian-eigenvector direction. Approximate that direction by initializing a random unit vector, taking the gradient with respect to r at a small displacement xi*d, normalizing it, and optionally repeating this power-iteration approximation. Scale the resulting direction by epsilon. For a model update, hold both the teacher prediction and the selected perturbation fixed while minimizing D(p_theta_hat(y|x)||p_theta(y|x+r_vadv)). Average this regularizer over available inputs to obtain R_VAT, and minimize supervised negative log likelihood plus alpha times R_VAT. The general formulation can use labeled and unlabeled inputs; the inspected semi-supervised driver applies its VAT branch to the unlabeled-input tensor. The pinned code defaults to one iteration, detaches the teacher and perturbation construction, and computes KL from softmax probabilities and stable log-softmax values. Its vatent branch separately adds conditional entropy minimization.