FedBE (Federated Bayesian Model Ensemble via Stochastic Weight Average Distillation)

A Bayesian server-side model aggregation algorithm for multi-round federated learning (Chen & Chao 2021, Sections 3-4, Equations 3-9, Algorithm 1) that treats client-updated models {w_i; i in S} as stochastic gradient trajectories to construct an approximate global model posterior p(w | D) on the server — either by fitting a SWAG-style diagonal Gaussian N(mu, Sigma_diag) with mu = sum_i (|D_i|/|D|) w_i and Sigma_diag = diag(sum_i (|D_i|/|D|) (w_i - mu)^2) (Equation 5) or a Dirichlet simplex convex-combination distribution w^{(m)} = sum_i (gamma_i^{(m)} |D_i| / sum_{i'} gamma_{i'}^{(m)} |D_{i'}|) w_i with gamma^{(m)} ~ Dir(alpha) (Equations 6-7) — samples M global models {w^{(m)} ~ p(w | D)}_{m=1}^M to form a Bayesian model ensemble {w_bar} U {w_i} U {w^{(m)}} that generates ensemble pseudo-labels p_hat_j = (1/M') sum_{m'=1}^{M'} p(y | x_j; w^{(m')}) (Equation 4) on an unlabeled server dataset U = {x_j}_{j=1}^J, and distills the pseudo-labeled set T = {(x_j, p_hat_j)}_{j=1}^J into a single student global model w using Stochastic Weight Averaging (SWA) with a cyclical learning rate schedule eta_SWA (Algorithm 1).