FedSR Simple Representation Regularization for Federated Domain Generalization

A decentralized federated domain generalization method (Nguyen, Torr, & Lim, 2022) that trains a probabilistic representation encoder p(z|x) = N(z; mu_{w_1}(x), sigma_{w_1}^2(x)) and predictive head p_hat(y|z) across K clients without sharing raw data, frequency spectra, or intermediate representations across clients. Each client i minimizes the composite local objective f_bar_i(w) + alpha^{L2R} ell_i^{L2R} + alpha^{CMI} ell_i^{CMI} (Eq. 9), where ell_i^{L2R} = E_{p_i(x)}[E_{p(z|x)}[||z||_2^2]] (Eq. 10) penalizes the L2 norm of the representation to implicitly align the marginal distribution p_i(z) toward a zero-mean Gaussian reference q(z) = N(0, sigma^2 I) (Eqs. 12-15), and ell_i^{CMI} = E_{p_i(x,y)}[KL[p(z|x) || r(z|y)]] (Eqs. 17-18) minimizes a tractable variational upper bound on the conditional mutual information I_i(x, z | y) (Lemma 1) using a shared learnable class-conditional Gaussian r(z|y) = N(z; mu_y, sigma_y^2), simultaneously upper-bounding E_{p_i(y)}[KL[p_i(z|y) || r(z|y)]] (Lemma 2, Eq. 19) to implicitly align class-conditional representation distributions across all clients.