FedMA (Federated Matched Averaging via Layer-Wise Bayesian Nonparametric Matching)

A layer-wise permutation-aware federated learning algorithm (FedMA, Algorithm 1) for modern neural networks (FC layers, CNN channels, and RNN/LSTM hidden states) that accounts for neuron permutation invariance W_1 Pi_j, Pi_j^T W_2 by matching and averaging hidden elements (neurons, convolutional channels, or recurrent states) bottom-up one layer at a time: at layer n, the server solves a Bayesian nonparametric Beta-Bernoulli Process MAP (BBP-MAP, Eq. 6) assignment problem via iterative Hungarian bipartite matching to align client weights {W_{j,n}}_{j=1}^J into global weights W_{G,n} = (1/J) sum_j W_{j,n} Pi_{j,n} (adaptively expanding layer width L in [max_j L_j, sum_j L_j] when distances exceed threshold epsilon), broadcasts W_{G,n}, and has clients permute layer n+1 via Pi_{j,n}^T W_{j,n+1} and retrain remaining layers n+1,...,N with layers 1,...,n frozen.