Home / Curated / Algorithms / LG-FedAvg (Local Global Federated Averaging)

LG-FedAvg (Local Global Federated Averaging)

Algorithms AIMech:LgFedAvgLocalGlobalRepresentationsFederatedLearning DRAFT

Aliases: LG-FedAvg, Local Global Federated Averaging, Federated Learning with Local and Global Representations

A personalized and communication-efficient federated learning algorithm (Liang, Liu, Ziyin, Allen, Auerbach, Brent, Salakhutdinov, & Morency 2020, Sections 3-4, Equations 3-7 & 36-38, Algorithm 1, Theorem 2, Corollary 1) that partitions each device m's model into lower-layer local representation parameters theta_m^l (which map raw heterogeneous or multimodal inputs X_m to compact feature representations ell_m(X_m; theta_m^l)) and shared upper-layer global classifier parameters theta^g (which produce predictions y_hat = g(ell_m(X_m; theta_m^l); theta^g)), jointly updates both theta_m^l and theta_m^g locally via E epochs of stochastic gradient descent (or minimax adversarial de-biasing min_{theta_m^l, theta_m^g} max_{phi_m} L_m^g(theta_m^l, theta_m^g) - alpha L_m^l(theta_m^l, phi_m) against protected attributes P_m, Equations 5-7), and communicates only the compact upper-layer global parameters theta_m^g to the server for sample-weighted averaging theta_{t+1}^g <- sum_{m in S_t} (N_m / N) theta_m^g.

Identity Notes & Concept Scope Distinct from FedPer (AIMech:FedPerFederatedLearningWithPersonalizationLayers, Arivazhagan et al. 2019) and FedRep (AIMech:FedRepExploitingSharedRepresentationsForPersonalizedFederatedLearning, Collins et al. 2021), which reverse the architectural split by keeping lower feature-extraction layers global (W_B) and top classification heads private on each client; in contrast, LG-FedAvg keeps lower feature-extraction encoders theta_m^l local (allowing modality-specific or adversarially de-biased encoders) and shares only the compact top global classifier theta^g across clients.

Claims & Grounded Evidence

4 Claims

Mechanism #r476-local-encoder-global-head-decomposition-and-adversarial-debiasing-mechanism

Each device m maintains private lower-layer representation parameters theta_m^l and shared upper-layer parameters theta_m^g, transmitting only theta_m^g to the server in each round (Algorithm 1); when local data includes nuisance or protected attributes P_m, device m jointly trains an adversarial discriminator d_m(ell_m(X_m); phi_m) via the minimax objective min_{theta_m^l, theta_m^g} max_{phi_m} [L_m^g(theta_m^l, theta_m^g) - alpha L_m^l(theta_m^l, phi_m)], whose saddle point satisfies H(P_m | ell_m(X_m; theta_m^l)) = H(P_m) with optimal value H(Y_m | X_m) - H(P_m) (Liang et al. 2020, Section 3.3, Equations 5-7, Proposition 3, & Appendix B).

supports Source: Think Locally, Act Globally: Federated Learning with Local and Global Representations · Locator: Section 3.2-3.3 (Algorithm 1, Equations 5-7), pp. 4-5, and Appendix B (Proposition 3, Equations 39-52), pp. 21-22

Scope #r476-bias-variance-generalization-dominance-scope

Under linear regression with cross-device task variance rho^2 and observation noise variance sigma^2 across M devices with N samples in dimension d, Theorem 2 proves that LG-FedAvg with interpolation weight alpha in [0, 1] has generalization error E_LG(f_alpha) = alpha^2 (d/N) sigma^2 + (1 - alpha)^2 ((M - 1)/M) rho^2 + (1 - alpha^2) (d / (M N)) sigma^2, and Corollary 1 establishes that alpha^* = rho^2 / (rho^2 + (d/N) sigma^2) strictly dominates both purely local learning (alpha = 1) and purely global FedAvg (alpha = 0) whenever rho^2, sigma^2 > 0 (Liang et al. 2020, Section 4, Theorem 2 & Corollary 1, Appendix A Equations 36-38).

supports Source: Think Locally, Act Globally: Federated Learning with Local and Global Representations · Locator: Section 4 (Propositions 1-2, Theorem 2, Corollary 1), pp. 6-7, and Appendix A (Equations 13-38), pp. 17-20

Application #r476-heterogeneous-vision-multimodal-and-fair-federated-learning-application

Applied to heterogeneous rotated MNIST/CIFAR, multimodal vision-and-language VQA across modality-heterogeneous clients, and adversarial de-biasing of protected attributes on federated facial recognition (Liang et al. 2020, Section 5, Tables 1-4, Figures 2-5).

supports Source: Think Locally, Act Globally: Federated Learning with Local and Global Representations · Locator: Section 5 (Experiments, Tables 1-4, Figures 2-5), pp. 7-10

Limitation #r476-stateful-local-encoder-requirement-for-new-devices-limitation

Because feature extractors theta_m^l are trained locally and never aggregated on the server, deploying the global head theta^g to a newly joined device requires either an ensemble of existing clients' local encoders or local labeled data to train a new local representation (Liang et al. 2020, Section 3.2 'Dealing with new devices', p. 5).

supports Source: Think Locally, Act Globally: Federated Learning with Local and Global Representations · Locator: Section 3.2 (Dealing with new devices), p. 5

⚠️ Curated Limitations & Negative Results

Does not learn a single shared backbone representation on the server, so inference on unseen clients requires either transferring an ensemble of local feature extractors or training a local encoder.
Requires all local encoders ell_m(X_m; theta_m^l) to output representations of the same dimension so the shared top classifier g(.; theta^g) can operate on them.

Artificial Intelligence Ontology (AIO) Grounding

Mapped AIO Classes

AIO Target Class	Mapping Relation	Mapping Rationale
`AIO:FederatedLearning` Federated Learning	broad_match	LG-FedAvg is a personalized and representation-partitioned federated learning algorithm within the broader FederatedLearning category.

Evidence Sources & Primary Literature

3 Sources

Source ID	Publication / Document	Type / Access	Reference & Checksum
`r476-primary-pdf`	Think Locally, Act Globally: Federated Learning with Local and Global Representations Paul Pu Liang, Terrance Liu, Liu Ziyin, Nicholas B. Allen, Randy P. Auerbach, David Brent, Ruslan Salakhutdinov, Louis-Philippe Morency	primary paper full_text	ID: https://arxiv.org/abs/2001.01523 sha256:6a46a1e52019…
`r476-arxiv-abs`	Think Locally, Act Globally: Federated Learning with Local and Global Representations (arXiv abstract) Paul Pu Liang, Terrance Liu, Liu Ziyin, Nicholas B. Allen, Randy P. Auerbach, David Brent, Ruslan Salakhutdinov, Louis-Philippe Morency	primary paper abstract_only	ID: https://arxiv.org/abs/2001.01523 sha256:982fd8188a08…
`r476-metadata-cr`	Think Locally, Act Globally: Federated Learning with Local and Global Representations (Crossref metadata) Paul Pu Liang, Terrance Liu, Liu Ziyin, Nicholas B. Allen, Randy P. Auerbach, David Brent, Ruslan Salakhutdinov, Louis-Philippe Morency	primary paper abstract_only	ID: https://arxiv.org/abs/2001.01523 sha256:7004751c7f2b…

Ontology & Grounding Topology

Open in interactive network graph →

Curation Provenance

Curator: Antigravity

Method: ai_assisted

Model: Gemini

Created: 2026-10-19T09:19:00Z

Updated: 2026-10-19T09:19:00Z

Source File: data/curated/LgFedAvgLocalGlobalRepresentationsFederatedLearning.yaml · Schema: curated_concept.yaml

Explore across X-Mech

Related computational & biological Mechs: Embedding Map Network Graph ProteinTraitsMech (ESM-2 pLM) DUFMech (Domain pLM) TraitMech CultureMech All 14 X-Mech Repositories