pFedLA and HeurpFedLA (Layer-Wised Hypernetwork Model Aggregation for Personalized Federated Learning)

A layer-wise personalized federated learning framework (Ma, Zhang, Guo, & Xu 2022, Section 3, Equations 1-12, Algorithms 1-2) that deploys a dedicated server-side hypernetwork HN_i(v_i; psi_i) with trainable client embedding vector v_i and hypernetwork weights psi_i for each client i in {1, ..., N} to generate an N x n column-stochastic layer-wise aggregation weight matrix alpha_i = [alpha_i^{l1}, alpha_i^{l2}, ..., alpha_i^{ln}] satisfying sum_{j=1}^N alpha_i^{ln, j} = 1 (Equations 3-4), synthesizes client i's personalized model by layer-wise weighted combination theta_bar_i^{ln} = sum_{j=1}^N theta_j^{ln} alpha_i^{ln, j} across all clients' intermediate layers {theta_j^{ln}} (Equations 5-6), updates v_i and psi_i at the server using the chain-rule surrogate gradients Delta v_i = (nabla_{v_i} theta_bar_i)^T Delta theta_i and Delta psi_i = (nabla_{psi_i} theta_bar_i)^T Delta theta_i driven by client i's local parameter displacement Delta theta_i = theta_i - theta_bar_i^{(t+1)} (Equations 8-11, Algorithm 1), and optionally retains the top-k highest self-weighted layers theta_bar_i^{retain} = AT_k{theta_bar_i^{l1}, ..., theta_bar_i^{ln} | alpha_i^{l1, i}, ..., alpha_i^{ln, i}} purely locally without server aggregation or transmission in its communication-efficient variant HeurpFedLA (Equation 12, Algorithm 2).