Home / Curated / Algorithms / FedCurv (Fisher-Information-Regularized Federated Learning for Non-IID Data)

FedCurv (Fisher-Information-Regularized Federated Learning for Non-IID Data)

Algorithms AIMech:FedCurvFisherInformationRegularizedFederatedLearning DRAFT

Aliases: FedCurv, Fisher-Information-Regularized Federated Learning, Federated Curvature Regularization, EWC-Regularized Federated Learning

A curvature-regularized federated learning algorithm (Shoham et al. 2019, Section 3, Equations 1-5) that mitigates catastrophic forgetting across statistically heterogeneous (non-IID) clients by augmenting each client s's local empirical loss L_s(theta) in round t with an Elastic Weight Consolidation (EWC) quadratic penalty weighted by the diagonal empirical Fisher information matrices diag(I_{t-1,j}) of all other participating clients j in S \ {s}: L~_{t,s}(theta) = L_s(theta) + lambda * sum_{j in S \ {s}} (theta - theta^hat_{t-1,j})^T diag(I_{t-1,j}) (theta - theta^hat_{t-1,j}) (Equation 4), decoupled for linear communication overhead into aggregated server statistics P_{t-1} = sum_{j in S} diag(I_{t-1,j}) and Q_{t-1} = sum_{j in S} diag(I_{t-1,j}) theta^hat_{t-1,j} via L~_{t,s}(theta) = L_s(theta) + lambda * theta^T (P_{t-1} - diag(I_{t-1,s})) theta - 2*lambda * theta^T (Q_{t-1} - diag(I_{t-1,s}) theta^hat_{t-1,s}) + const (Equation 5).

Identity Notes & Concept Scope Distinct from FedProx (AIMech:FedProxProximalHeterogeneousFederatedOptimization, Li et al. 2020), which penalizes isotropic Euclidean distance (mu/2)*||theta - w^{t-1}||_2^2 around the scalar-averaged global model without parameter-wise Fisher curvature weighting or peer-model decoupling; and distinct from FFL-ERL / Fisher-weighted server model fusion, as FedCurv applies the peer-summed diagonal Fisher quadratic regularizer directly during local client SGD optimization via aggregated server vectors (P_{t-1}, Q_{t-1}).

Claims & Grounded Evidence

4 Claims

Mechanism #r452-decoupled-fisher-quadratic-penalty-mechanism

Rewrites the pairwise peer Fisher penalty sum_{j in S \ {s}} (theta - theta^hat_{t-1,j})^T diag(I_{t-1,j}) (theta - theta^hat_{t-1,j}) (Equation 4) into the algebraically identical decoupled form theta^T (P_{t-1} - diag(I_{t-1,s})) theta - 2 * theta^T (Q_{t-1} - diag(I_{t-1,s}) theta^hat_{t-1,s}) + const (Equation 5), reducing per-round communication from O(|S|) vectors per client to exactly 3 parameter-sized vectors (model parameters plus two Fisher summary vectors) independent of the number of clients.

supports Source: Overcoming Forgetting in Federated Learning on Non-IID Data · Locator: Section 3 Equations 4-5, p. 3

Scope #r452-laplace-posterior-multi-task-scope

Interprets each communication round of non-IID federated learning as distributed multi-task continual learning under a diagonal Laplace approximation of the joint log-posterior log p(theta | D) (Equations 1-3), preventing local SGD steps with large local epoch counts E from overwriting parameters with high Fisher information on other clients' data.

supports Source: Overcoming Forgetting in Federated Learning on Non-IID Data · Locator: Section 3 Equations 1-4 & Section 4 Experimental Results, pp. 2-5

Limitation #r452-diagonal-fisher-approximation-and-communication-factor-limitation

Approximates each client's local Hessian by the diagonal of the empirical Fisher information matrix diag(I_{t,s}), omitting off-diagonal parameter correlations, and increases per-round communication bandwidth by a factor of up to 3 compared to vanilla FedAvg (reduced to 2 in round-initial Fisher evaluation variants).

supports Source: Overcoming Forgetting in Federated Learning on Non-IID Data · Locator: Section 3 Bandwidth Reduction Discussion & Section 5 Conclusions, pp. 3-5

Application #r452-non-iid-classification-communication-reduction-application

Applied to non-IID federated image classification where each client holds examples from only a strict subset of classes, reducing the communication rounds required to reach target test accuracy under large local epoch budgets.

supports Source: Overcoming Forgetting in Federated Learning on Non-IID Data · Locator: Section 4 Experimental Results (Figures 1-3, Table 1), pp. 4-5

⚠️ Curated Limitations & Negative Results

Relies on a diagonal empirical Fisher approximation of the local Hessian, neglecting off-diagonal parameter interactions.
Transmits two additional parameter-sized Fisher summary vectors per round (3x total communication volume per round in the standard end-of-round Fisher variant).

Artificial Intelligence Ontology (AIO) Grounding

Mapped AIO Classes

AIO Target Class	Mapping Relation	Mapping Rationale
`AIO:FederatedLearning` Federated Learning	narrow_match	FedCurv is a specific federated learning algorithm that regularizes local client updates using peer clients' diagonal Fisher information matrices.

Evidence Sources & Primary Literature

3 Sources

Source ID	Publication / Document	Type / Access	Reference & Checksum
`r452-primary-pdf`	Overcoming Forgetting in Federated Learning on Non-IID Data Neta Shoham, Tomer Avidor, Aviv Keren, Nadav Israel, Daniel Benditkis, Liron Mor-Yosef, Itai Zeitak	primary paper full_text	ID: https://arxiv.org/abs/1910.07796 sha256:47b9944148ef…
`r452-arxiv-abs`	Overcoming Forgetting in Federated Learning on Non-IID Data (arXiv abstract) Neta Shoham, Tomer Avidor, Aviv Keren, Nadav Israel, Daniel Benditkis, Liron Mor-Yosef, Itai Zeitak	primary paper abstract_only	ID: https://arxiv.org/abs/1910.07796 sha256:cdc3282809a3…
`r452-metadata-cr`	Overcoming Forgetting in Federated Learning on Non-IID Data (Crossref metadata) Neta Shoham, Tomer Avidor, Aviv Keren, Nadav Israel, Daniel Benditkis, Liron Mor-Yosef, Itai Zeitak	primary paper abstract_only	ID: https://arxiv.org/abs/1910.07796 sha256:3d7b485bcbe9…

Ontology & Grounding Topology

Open in interactive network graph →

Curation Provenance

Curator: Antigravity

Method: ai_assisted

Model: Gemini

Created: 2026-10-19T08:55:00Z

Updated: 2026-10-19T08:55:00Z

Source File: data/curated/FedCurvFisherInformationRegularizedFederatedLearning.yaml · Schema: curated_concept.yaml

Explore across X-Mech

Related computational & biological Mechs: Embedding Map Network Graph ProteinTraitsMech (ESM-2 pLM) DUFMech (Domain pLM) TraitMech CultureMech All 14 X-Mech Repositories