FedMGDA+ (Multi-Objective Pareto-Stationary Federated Learning with Normalized Multiple Gradient Descent)

A multi-objective federated optimization algorithm (Hu, Shaloudegi, Zhang, & Yu 2022, Sections 3-5, Definition 1, Lemma 1, Equations 4-9 & 19-30, Algorithm 1, Theorems 1a, 1b, & 2) that formulates federated learning over m users as vector minimization min_{w in R^d} f(w) = (f_1(w), ..., f_m(w)) targeting Pareto-stationary solutions w^* where min_{lambda in Delta} ||J_f(w^*) lambda||_2 = 0 (Definition 1, Equation 11), while interpreting intermediate-k local-epoch FedAvg and FedProx through the proximal average A_{f, lambda}^eta(w) whose proximal map equals sum_{i=1}^m lambda_i P_{f_i}^eta(w) (Equations 4-9). In each communication round t, sampled clients i in I_t perform k local epochs of r mini-batch steps and return pseudo-gradients g_i = w^0 - w, which the server normalizes to unit length g_bar_i = g_i / ||g_i||_2 (Algorithm 1, line 5) and combines by solving the epsilon-constrained minimum-norm quadratic program over the probability simplex lambda^* = argmin_{lambda in Delta, ||lambda - lambda_0||_infty <= epsilon} ||sum_{i in I_t} lambda_i g_bar_i||_2^2 to form the common descent direction d_t = sum_{i in I_t} lambda_i^* g_bar_i and global update w_{t+1} = w_t - eta_t d_t (Equation 25, Algorithm 1 lines 6-9), interpolating continuously between normalized FedAvg (epsilon = 0) and unconstrained MGDA (epsilon = 1) while guaranteeing simultaneous descent for all participating clients and invariance against both additive loss-bias and positive multiplicative loss-scaling attacks.