Bayesian Data Augmentation for Deep Models

Treat each additional labeled example z=(t_a,x_a) as a latent variable and seek a maximum-a-posteriori parameter estimate, rather than sampling a posterior over all network weights. At an iteration, sample synthetic labeled data using the current model and approximate the expected augmented log posterior with a Monte Carlo average. The generalized M-step seeks parameters that increase this sampled objective rather than solving a global maximization exactly. The published objective includes a parameter prior, an average of real-data label/data log terms and a separate average of synthetic-data label/data log terms. In the paper’s concrete GAN realization, a class-conditioned generator produces labeled images, a classifier learns dataset labels on real and generated examples, and a separate authenticator distinguishes real from generated images. The published discriminative step minimizes J_C+J_AG, while the generative step minimizes J_C−J_AG, combining class consistency with adversarial realism. J_C separately averages the real and synthetic classification losses, rather than averaging one pooled batch with weights determined solely by its real-to-synthetic count ratio. The practical training procedure uses minibatches and one epoch of stochastic updates per parameter-update iteration. The GAN architecture is one realization of the augmentation framework, not a requirement for every possible realization.