Information maximizing self-augmented training

For a current network snapshot, hold the original-input output distribution fixed and minimize cross-entropy against the network prediction on a transformed input, summed over discrete output components and averaged over samples. Transformations include random perturbations or virtual adversarial perturbations approximately chosen to maximize local prediction change. For categorical clustering use R_SAT−lambda[H(Y)−H(Y|X)], with the marginal obtained by averaging conditional predictions. The nonuniform-prior variant instead minimizes R_SAT+lambda H(Y|X) subject to KL(p(Y)||q(Y))≤delta, implemented with an increasing positive-part penalty. For binary hashing the factorized conditional model uses the truncated information objective sum_d I(X;Y_d)−sum_(d≠e) I(Y_d;Y_e), with ordered distinct bit pairs. The pinned hash code enumerates unordered pairs and multiplies their mutual-information sum by two, confirming that convention. Its clustering code detaches the teacher and implements epsilon-stabilized KL consistency. Ideal KL and cross-entropy have the same parameter gradient for a fixed teacher; the code adds epsilon inside the logarithm and therefore only approximates that unstabilized identity. That code defaults to H(Y|X)−4H(Y), a weighted entropy variant rather than the unit-weight information expression.