FedFA (Federated Feature Augmentation and Momentum-Queued Feature Anchor Loss)

A federated learning algorithm for heterogeneous client distributions (Zhou & Konukoglu 2023, Sections 3.1-3.3, Equations 4-15, Lemma 1, Algorithm 1) that integrates two complementary mechanisms: (1) Probabilistic Federated Feature Augmentation (FFA, Equations 4-8), which inserts a parameter-free layer after each network block to normalize channel/token activations x_hat = (x - mu) / (sigma + epsilon) and re-modulate them with Gaussian-perturbed statistics y = gamma o x_hat + beta where gamma ~ N(sigma, Sigma_sigma^2) and beta ~ N(mu, Sigma_mu^2), estimating variances online via EMA buffers Sigma_mu^{(t)} = alpha Sigma_mu^{(t-1)} + (1 - alpha) gamma_mu^{(t)} Sigma_{mu,B}^{(t)} with variance-proportional channel-adaptive weights gamma_{\cdot,c}^{(t)} = D * Sigma_{\cdot,B,c}^{(t)} / sum_{i=1}^D Sigma_{\cdot,B,i}^{(t)} (Equation 8) and resetting local variance buffers each round to the aggregated server-side buffer Sigma_*^{(r,0)} = (1 / |S_r|) sum_{k in S_r} Sigma_k^{(r,E)}; and (2) Momentum-Queued Federated Feature Anchor Loss (FFAL, Equations 9-14), which aligns final features f_i and momentum-queued historical features q_{c,m} in Q_k (|Q_k| = C x M) with shared global class anchors A^{(r)} via the symmetric cosine-distance loss L_anchor(f_i, a_{y_i}) = 1 - cos(f_i, a_{y_i}) + (1 / M) sum_{m=1}^M (1 - cos(f_i, q_{y_i,m})) (Equation 11, satisfying Lemma 1 E[1 - cos(f, q_{c,m})] = 1 - cos(f, a_c) under a von Mises-Fisher distribution with mean direction a_c) and updates client anchors via EMA before class-specific server aggregation a_c^{(r+1)} = (1 / |S_{r,c}|) sum_{k in S_{r,c}} a_{k,c}^{(r,E)} (Equation 13).