Home / Curated / Algorithms / FedBN (Federated Learning on Non-IID Features via Local Batch Normalization)

FedBN (Federated Learning on Non-IID Features via Local Batch Normalization)

Algorithms AIMech:FedBnFederatedLearningOnNonIidFeaturesViaLocalBatchNormalization DRAFT

Aliases: FedBN, Federated Learning with Local Batch Normalization, Federated Learning on Non-IID Features via Local Batch Normalization

A federated learning aggregation modality (FedBN, Algorithm 1) designed for feature-shift non-IID data (covariate shift where marginal input distributions P_i(x) vary across clients with shared conditional P(y|x), or concept shift in P_i(x|y) with shared P(y)), in which every client updates all neural network parameters locally for E steps, but during server-side synchronization at t mod E == 0 all Batch Normalization (BN) layer parameters (affine scale and shift parameters gamma, beta and running mean/variance statistics) are strictly excluded from global averaging and retained locally on each client while all non-BN (convolutional and fully connected) layer weights w_{k,l} are averaged.

Identity Notes & Concept Scope Distinct from FedProx (AIMech:FedProxProximalHeterogeneousFederatedOptimization) and SCAFFOLD (AIMech:ScaffoldStochasticControlledAveragingForFederatedLearning), which synchronize all model layers including Batch Normalization parameters and add proximal or control-variate corrections, and from FedRep (AIMech:FedRepExploitingSharedRepresentationsForPersonalizedFederatedLearning, which keeps the top classification head h_i local to address heterogeneous label conditional distributions P_i(y|x) rather than keeping internal Batch Normalization layers local to mitigate feature shift P_i(x)).

Claims & Grounded Evidence

4 Claims

Mechanism #r406-ntk-block-diagonal-gram-matrix-and-strictly-faster-convergence-rate

In the over-parameterized two-layer ReLU network regime with Batch Normalization under heterogeneous client input covariances S_i = E[x^i (x^i)^T] (Assumption 4.1), Lemma 4.3 and Definition 4.2 show that FedBN induces a block-diagonal auxiliary NTK magnitude Gram matrix G^{*infty} = diag(G_1^infty,...,G_N^infty) compared to the full N*M x N*M Gram matrix G^infty of FedAvg, and Theorem 4.4 with Corollaries 4.5-4.6 proves that mu_0^* = lambda_{min}(G^{*infty}) = min_i lambda_{min}(G_i^infty) >= lambda_{min}(G^infty) = mu_0 > 0, yielding a strictly faster exponential convergence factor (1 - eta * mu_0^* / 2)^t <= (1 - eta * mu_0 / 2)^t.

supports Source: FedBN: Federated Learning on Non-IID Features via Local Batch Normalization · Locator: Section 4 Convergence Analysis (Assumption 4.1, Definition 4.2, Lemma 4.3, Theorem 4.4, Corollaries 4.5-4.6) & Appendix B, pp. 4-6, 13-15

Scope #r406-zero-overhead-compatibility-with-fedprox-and-domain-adaptation

FedBN requires zero additional communication or hyperparameter tuning beyond standard FedAvg (slightly reducing communicated payload by omitting BN parameters), composes orthogonally with optimization regularizers such as FedProx, and enables adaptation to unseen target client domains by recomputing local BN statistics on unlabeled target batches.

supports Source: FedBN: Federated Learning on Non-IID Features via Local Batch Normalization · Locator: Section 3.2 Algorithm 1 & Section 5.3-5.4 Experiments and Analysis (Figures 3-4, Table 1), pp. 3-4, 7-8

Limitation #r406-batch-normalization-architecture-and-feature-shift-scope-limitation

FedBN applies only to neural architectures equipped with Batch Normalization layers (which require local mini-batches large enough to estimate stable batch statistics) and specifically targets feature-shift non-IID regimes (P_i(x) variation) rather than severe concept-shift where clients have conflicting label assignments P_i(y|x) for identical features.

supports Source: FedBN: Federated Learning on Non-IID Features via Local Batch Normalization · Locator: Section 3.1 Problem Statement & Section 4.1 Setup and Key Lemmas, pp. 2-5

Application #r406-multi-domain-digits-office-caltech-domainnet-and-abide-neuroimaging-application

Evaluated on benchmark multi-domain vision datasets ( five-source Digits: MNIST, SVHN, USPS, SynthDigits, MNIST-M; four-source Office-Caltech10; six-source DomainNet) and multi-site fMRI neuroimaging classification (ABIDE I across 4 clinical institutions), consistently outperforming Single-set, FedAvg, and FedProx.

supports Source: FedBN: Federated Learning on Non-IID Features via Local Batch Normalization · Locator: Section 5 Experiments (Figures 2-4, Tables 1-2), pp. 6-8

⚠️ Curated Limitations & Negative Results

Applicable only to neural networks with Batch Normalization layers and sufficient local batch size to estimate batch statistics.
Addresses feature-shift non-IID distributions rather than conflicting label-conditional distributions P_i(y|x).

Artificial Intelligence Ontology (AIO) Grounding

Mapped AIO Classes

AIO Target Class	Mapping Relation	Mapping Rationale
`AIO:FederatedLearning` Federated Learning	narrow_match	FedBN is a FederatedLearning aggregation scheme that keeps Batch Normalization parameters local to handle feature shift across clients.

Evidence Sources & Primary Literature

3 Sources

Source ID	Publication / Document	Type / Access	Reference & Checksum
`r406-primary-pdf`	FedBN: Federated Learning on Non-IID Features via Local Batch Normalization Xiaoxiao Li, Meirui Jiang, Xiaofei Zhang, Michael Kamp, Qi Dou	primary paper full_text	ID: https://openreview.net/forum?id=6YEQUn0QICG sha256:47b20c25d772…
`r406-arxiv-abs`	FedBN: Federated Learning on Non-IID Features via Local Batch Normalization (arXiv abstract) Xiaoxiao Li, Meirui Jiang, Xiaofei Zhang, Michael Kamp, Qi Dou	primary paper abstract_only	ID: https://arxiv.org/abs/2102.07623 sha256:17d7f818cc9d…
`r406-metadata-cr`	FedBN: Federated Learning on Non-IID Features via Local Batch Normalization (Crossref metadata) Xiaoxiao Li, Meirui Jiang, Xiaofei Zhang, Michael Kamp, Qi Dou	primary paper abstract_only	ID: https://openreview.net/forum?id=6YEQUn0QICG sha256:9cc870f1447c…

Ontology & Grounding Topology

Open in interactive network graph →

Curation Provenance

Curator: Antigravity

Method: ai_assisted

Model: Gemini

Created: 2026-10-19T08:09:00Z

Updated: 2026-10-19T08:09:00Z

Source File: data/curated/FedBnFederatedLearningOnNonIidFeaturesViaLocalBatchNormalization.yaml · Schema: curated_concept.yaml

Explore across X-Mech

Related computational & biological Mechs: Embedding Map Network Graph ProteinTraitsMech (ESM-2 pLM) DUFMech (Domain pLM) TraitMech CultureMech All 14 X-Mech Repositories