KT-pFL (Parameterized Knowledge Transfer for Personalized Federated Learning)

A personalized group knowledge transfer federated learning algorithm (Zhang, Guo, Ma, Wang, Xu, & Wu 2021, Sections 3-4, Definitions 3.1-3.2, Equations 2-7, Algorithm 1, Theorem 4.1, Remark 4.1) that enables clients n in {1, ..., N} with heterogeneous or homogeneous model architectures w^n and non-IID private datasets D_n to collaboratively learn personalized models via a shared public reference dataset D_r and a trainable N x N knowledge coefficient matrix c in R^{N x N} (Equation 3). Each client n minimizes the personalized loss L_{per,n}(w^n) = L_n(w^n) + lambda sum_{x_hat in D_r} L_{KL}(sum_{m=1}^N c_{mn} s(w^m, x_hat), s(w^n, x_hat)) where s(w^m, x_hat) = softmax(z^m(x_hat)/T) (Definition 3.1, Equation 2), while the system jointly minimizes min_{w,c} L(w, c) = sum_{n=1}^N (D_n/D) L_{per,n}(w^n) + rho ||c - 1/N||_2^2 with Frobenius shrinkage toward the uniform collaboration matrix 1/N (Equation 4) via alternating optimization: (1) client-side local SGD on D_n (Equation 5) followed by public-data KL distillation toward client n's personalized server soft prediction sum_{m=1}^N c_{mn} s(w^m, xi_r) (Equation 6), and (2) server-side gradient descent on c via c <- c - eta_3 lambda sum_{n=1}^N (D_n/D) nabla_c L_{KL}(sum_{m=1}^N c_m s(w^{m,*}, xi_r), s(w^{n,*}, xi_r)) - 2 eta_3 rho (c - 1/N) (Equation 7).