Home / Curated / Algorithms / FedDyn Dynamic Regularization for Federated Learning

FedDyn Dynamic Regularization for Federated Learning

Algorithms AIMech:FedDynDynamicRegularizationForFederatedLearning DRAFT

Aliases: FedDyn, Federated Learning Based on Dynamic Regularization, Dynamic Regularized Federated Optimization

A communication-efficient federated optimization method (FedDyn) in which each active device k in P_t at round t minimizes a dynamically regularized local risk objective theta_k^t = argmin_theta [ L_k(theta) - <nabla L_k(theta_k^{t-1}), theta> + (alpha / 2) ||theta - theta^{t-1}||^2 ], recursively updates its local gradient state via the exact first-order optimality relation nabla L_k(theta_k^t) = nabla L_k(theta_k^{t-1}) - alpha * (theta_k^t - theta^{t-1}), and transmits only theta_k^t to the server, which updates a server state vector h^t = h^{t-1} - (alpha / m) * (sum_{k in P_t} theta_k^t - P * theta^{t-1}) == (1/m) sum_{k=1}^m nabla L_k(theta_k^t) and forms the next global model theta^t = (1/P) sum_{k in P_t} theta_k^t - (1/alpha) * h^t, ensuring that local optima dynamically align with stationary points of the global empirical loss L(theta) = (1/m) sum_{k=1}^m L_k(theta) under arbitrary statistical heterogeneity and partial participation without extra communication payload per round.

Identity Notes & Concept Scope Distinct from FedProx (AIMech:FedProxProximalHeterogeneousFederatedOptimization, which penalizes (mu/2)||theta - theta^{t-1}||^2 without the dynamic linear gradient-alignment term -<nabla L_k(theta_k^{t-1}), theta>, so client minimizers do not align with theta* when nabla L_k(theta*) != 0), from SCAFFOLD (AIMech:ScaffoldStochasticControlledAveragingForFederatedLearning, which doubles communication payload by transmitting control variate updates Delta c_i alongside model parameters), and from FedNova (AIMech:FedNovaNormalizedAveragingForHeterogeneousFederatedOptimization). No pinned AIO class covers FedDyn.

Claims & Grounded Evidence

4 Claims

Mechanism #r387-dynamic-regularization-stationary-alignment-and-h-invariant

By augmenting L_k(theta) with -<nabla L_k(theta_k^{t-1}), theta> + (alpha/2)||theta - theta^{t-1}||^2, exact local minimization enforces nabla L_k(theta_k^t) - nabla L_k(theta_k^{t-1}) + alpha(theta_k^t - theta^{t-1}) = 0 (Eq. 2), so the server recursion h^t = h^{t-1} - (alpha/m)(sum_{k in P_t} theta_k^t - P * theta^{t-1}) maintains the exact invariant h^t = (1/m) sum_{k=1}^m nabla L_k(theta_k^t) at every round t >= 1 even under partial participation P < m, and any fixed point theta_k^infty = theta^infty = theta* satisfies (1/m) sum_{k=1}^m nabla L_k(theta*) = 0.

supports Source: Federated Learning Based on Dynamic Regularization · Locator: Section 2.2 FedDyn Method, Algorithm 1, Equations 1-3, and Appendix A.1 Lemma 2, pp. 3-5, 13

Mechanism #r387-communication-round-convergence-rates

Under P-out-of-m partial client participation and L-smooth local losses, Theorems 1, 2, and 3 prove that FedDyn achieves O((1/T) sqrt(m/P) (L * D^2 + (1/L) * zeta_*^2)) convergence for convex losses, linear convergence (1 + mu / alpha)^{-(T-1)} with alpha = max(5*(m/P)*mu, 30*L) (i.e. O((m/P)*(L/mu) log(1/eps)) communication rounds independent of heterogeneity zeta_*^2) for mu-strongly convex losses, and O((L/T)*(m/P)) ergodic gradient norm decay for nonconvex losses.

supports Source: Federated Learning Based on Dynamic Regularization · Locator: Section 3 Theorems 1, 2, 3, and 4, and Table 1, pp. 5-6

Limitation #r387-stateful-client-gradient-and-exact-subproblem-limitation

Requires each client k to maintain a persistent local gradient state nabla L_k(theta_k^{t-1}) in R^d across communication rounds (making it unsuitable for purely stateless cross-device settings where clients participate only once) and assumes exact or high-accuracy minimization of the local regularized subproblem.

supports Source: Federated Learning Based on Dynamic Regularization · Locator: Section 2.2 Algorithm 1 and Section 3 Theorems 1-3 assumptions, pp. 4-6

Application #r387-heterogeneous-federated-vision-and-nlp-application

Applied to federated classification and language modeling on CIFAR-10, CIFAR-100, MNIST, Fashion-MNIST, Shakespeare, and StackOverflow across IID, Dirichlet non-IID, and 10%/1% partial participation regimes, achieving target accuracy in 3x-5x fewer communication rounds than FedAvg, FedProx, and SCAFFOLD.

supports Source: Federated Learning Based on Dynamic Regularization · Locator: Section 4 Experiments, Tables 2-3 & Figure 1, pp. 6-8

⚠️ Curated Limitations & Negative Results

Requires stateful clients that store nabla L_k(theta_k^{t-1}) in R^d between rounds of participation.
Theoretical guarantees assume exact minimization of the local dynamically regularized objective at each active client.

Artificial Intelligence Ontology (AIO) Grounding

Mapped AIO Classes

AIO Target Class	Mapping Relation	Mapping Rationale
`AIO:FederatedLearning` Federated Learning	narrow_match	FedDyn is a federated learning optimization algorithm that dynamically modifies client risk objectives with a linear gradient-alignment term and quadratic proximal penalty.

Evidence Sources & Primary Literature

3 Sources

Source ID	Publication / Document	Type / Access	Reference & Checksum
`r387-primary-pdf`	Federated Learning Based on Dynamic Regularization Durmus Alp Emre Acar, Yue Zhao, Ramon Matas Navarro, Matthew Mattina, Paul N. Whatmough, Venkatesh Saligrama	primary paper full_text	ID: https://openreview.net/forum?id=B7v4QMR6Z9w sha256:b953517a3bc4…
`r387-arxiv-abs`	Federated Learning Based on Dynamic Regularization (arXiv abstract) Durmus Alp Emre Acar, Yue Zhao, Ramon Matas Navarro, Matthew Mattina, Paul N. Whatmough, Venkatesh Saligrama	primary paper abstract_only	ID: https://arxiv.org/abs/2111.04263 sha256:e9fca89d3d29…
`r387-metadata-cr`	Federated Learning Based on Dynamic Regularization (Crossref metadata) Durmus Alp Emre Acar, Yue Zhao, Ramon Matas Navarro, Matthew Mattina, Paul N. Whatmough, Venkatesh Saligrama	primary paper abstract_only	ID: https://openreview.net/forum?id=B7v4QMR6Z9w sha256:de824877494e…

Ontology & Grounding Topology

Open in interactive network graph →

Curation Provenance

Curator: Antigravity

Method: ai_assisted

Model: Gemini

Created: 2026-10-19T07:50:00Z

Updated: 2026-10-19T07:50:00Z

Source File: data/curated/FedDynDynamicRegularizationForFederatedLearning.yaml · Schema: curated_concept.yaml

Explore across X-Mech

Related computational & biological Mechs: Embedding Map Network Graph ProteinTraitsMech (ESM-2 pLM) DUFMech (Domain pLM) TraitMech CultureMech All 14 X-Mech Repositories