FedEMA (Federated Divergence-Aware Exponential Moving Average Update for Self-Supervised Learning)

A federated self-supervised representation learning algorithm (Zhuang, Wen, & Zhang 2022, Sections 3.2 & 4, Equations 1-3, Algorithm 1) built on a generalized Siamese-network Federated Self-Supervised Learning (FedSSL) framework in which each client k in {1, ..., K} trains an online network W_k^o = (W_k, W_k^p) (online encoder W_k and MLP predictor W_k^p) and a momentum target encoder W_k^t updated per mini-batch via W_k^t <- m W_k^t + (1 - m) W_k on unlabeled private data D_k, while the server aggregates online encoders and predictors via W_g^{r+1} = sum_{k in S_t} (n_k/n) W_k^r and W_g^{p,r+1} = sum_{k in S_t} (n_k/n) W_k^{p,r}. Instead of overwriting client k's online network with the global network (FedBYOL), FedEMA retains local non-IID representation knowledge by updating client k's online encoder and predictor at round r via a divergence-aware exponential moving average W_k^r = mu W_k^{r-1} + (1 - mu) W_g^r (Equation 1) and W_k^{p,r} = mu W_k^{p,r-1} + (1 - mu) W_g^{p,r} (Equation 2), where the dynamic decay rate mu = min(lambda_k ||W_g^r - W_k^{r-1}||_2, 1) (Equation 3) scales with the L2 divergence between the global encoder W_g^r and client k's previous online encoder W_k^{r-1}, and the client-specific scaler lambda_k = tau / ||W_g^{r+1} - W_k^r||_2 with expected initial decay tau in [0.5, 1) is automatically calibrated once by an autoscaler at the earliest round r client k is sampled (Algorithm 1, line 10).