Home / Curated / Algorithms / FedPara and pFedPara (Low-Rank Hadamard Product Parameterization for Federated Learning)

FedPara and pFedPara (Low-Rank Hadamard Product Parameterization for Federated Learning)

Algorithms AIMech:FedParaAndPFedParaLowRankHadamardProductParameterization DRAFT

Aliases: FedPara and pFedPara, FedPara, pFedPara, Low-Rank Hadamard Product Federated Parameterization

A communication-efficient layer parameterization for federated learning and its personalized variant (Hyeon-Woo, Ye-Bin, & Oh 2022, Section 2, Equations 1-4, Propositions 1-3) that expresses each weight matrix W in R^{m x n} as the Hadamard (element-wise) product of two low-rank matrices W = W_1 odot W_2 = (X_1 Y_1^T) odot (X_2 Y_2^T) with X_1 in R^{m x r_1}, Y_1 in R^{n x r_1}, X_2 in R^{m x r_2}, and Y_2 in R^{n x r_2} (and 4D convolutional kernels via Tucker mode-1/mode-2 products W_conv = (T_1 x_1 X_1 x_2 Y_1) odot (T_2 x_1 X_2 x_2 Y_2)), transmitting only the low-rank factors during federated aggregation; in pFedPara (Section 2.3, Equation 4), W = W_1 odot (W_2 + 1) = W_1 odot W_2 + W_1 transfers only the global factor W_1 to the server while keeping W_2 strictly local on each client.

Identity Notes & Concept Scope Distinct from PowerSGD (AIMech:PowerSgdLowRankGradientCompressionWithErrorFeedback, Vogels et al. 2019), which compresses full-model gradients via power-iteration low-rank approximations with error feedback rather than reparameterizing the model weights via an unconstrained-rank Hadamard product of low-rank factors, and from FedRep (AIMech:FedRepExploitingSharedRepresentationsForPersonalizedFederatedLearning, Collins et al. 2021), which splits personalization by network depth (shared body vs local head) rather than intra-layer Hadamard decomposition W_1 odot (W_2 + 1).

Claims & Grounded Evidence

4 Claims

Mechanism #r439-hadamard-rank-multiplication-and-minimal-parameter-propositions

By Proposition 1, for W_1 = X_1 Y_1^T of rank <= r_1 and W_2 = X_2 Y_2^T of rank <= r_2, the Hadamard product W = W_1 odot W_2 satisfies rank(W) <= r_1 r_2 and achieves full rank min(m, n) whenever r_1 r_2 >= min(m, n); by Proposition 2, for any fixed target rank bound R^2 = r_1 r_2, choosing r_1 = r_2 = R uniquely minimizes the total parameter count to 2 R (m + n), and Proposition 3 extends the same rank bound rank(W^{(1)}) = rank(W^{(2)}) <= R^2 to 4D convolutional tensors with 2 R (O + I + R K_1 K_2) parameters.

supports Source: FedPara: Low-Rank Hadamard Product for Communication-Efficient Federated Learning · Locator: Section 2.1-2.2 Low-Rank Hadamard Product and Convolutional Extension (Propositions 1, 2, and 3) & Appendix A Proofs

Mechanism #r439-pfedpara-global-personal-additive-multiplicative-decomposition

In pFedPara (Section 2.3, Eq. 4), parameterizing each layer as W = W_1 odot (W_2 + 1) = W_1 + W_1 odot W_2 decomposes the weight matrix into an additive global low-rank component W_glo = W_1 (aggregated across clients by the server) and a client-specific residual W_per = W_1 odot W_2 modulated by the local factor W_2, halving communication cost relative to FedPara while preserving rank(W) <= r_1 (r_2 + 1).

supports Source: FedPara: Low-Rank Hadamard Product for Communication-Efficient Federated Learning · Locator: Section 2.3 Personalized FedPara (pFedPara, Equation 4) & Section 3.3 Personalized FL Evaluation

Limitation #r439-bilinear-factor-nonconvexity-and-small-kernel-overhead-limitation

Introduces four-factor multilinear product landscapes during local gradient updates and requires 2 R (O + I + R K_1 K_2) < O I K_1 K_2 (i.e., R <= sqrt(O I / 2)) to achieve parameter savings on convolutional layers.

supports Source: FedPara: Low-Rank Hadamard Product for Communication-Efficient Federated Learning · Locator: Section 2.1 Property 1 & Section 2.2 Convolutional Layer Extension

Application #r439-communication-efficient-vgg-resnet-and-lstm-federated-training-application

Applied to communication-efficient federated training of VGG16, ResNet-18, and LSTM models on CIFAR-10, CIFAR-100, CINIC-10, and Shakespeare under IID and non-IID partitions, reducing communication costs by 3x to 10x at matching or higher accuracy.

supports Source: FedPara: Low-Rank Hadamard Product for Communication-Efficient Federated Learning · Locator: Section 3 Experiments (Figures 3-6, Tables 1-3)

⚠️ Curated Limitations & Negative Results

Requires reconstructing W = (X_1 Y_1^T) odot (X_2 Y_2^T) during forward and backward passes.
Parameter compression requires choosing rank hyperparameter R < sqrt(m n / (2(m + n))).

Artificial Intelligence Ontology (AIO) Grounding

Mapped AIO Classes

AIO Target Class	Mapping Relation	Mapping Rationale
`AIO:FederatedLearning` Federated Learning	narrow_match	FedPara and pFedPara are communication-efficient and personalized federated learning methods based on low-rank Hadamard weight parameterization.

Evidence Sources & Primary Literature

3 Sources

Source ID	Publication / Document	Type / Access	Reference & Checksum
`r439-primary-pdf`	FedPara: Low-Rank Hadamard Product for Communication-Efficient Federated Learning Nam Hyeon-Woo, Moon Ye-Bin, Tae-Hyun Oh	primary paper full_text	ID: https://openreview.net/forum?id=d71n4ftoCBy sha256:1ae6352c22b2…
`r439-arxiv-abs`	FedPara: Low-Rank Hadamard Product for Communication-Efficient Federated Learning (arXiv abstract) Nam Hyeon-Woo, Moon Ye-Bin, Tae-Hyun Oh	primary paper abstract_only	ID: https://arxiv.org/abs/2108.06098 sha256:2ce8fd75c192…
`r439-metadata-cr`	FedPara: Low-Rank Hadamard Product for Communication-Efficient Federated Learning (Crossref metadata) Nam Hyeon-Woo, Moon Ye-Bin, Tae-Hyun Oh	primary paper abstract_only	ID: https://openreview.net/forum?id=d71n4ftoCBy sha256:239bffd350ab…

Ontology & Grounding Topology

Open in interactive network graph →

Curation Provenance

Curator: Antigravity

Method: ai_assisted

Model: Gemini

Created: 2026-10-19T08:42:00Z

Updated: 2026-10-19T08:42:00Z

Source File: data/curated/FedParaAndPFedParaLowRankHadamardProductParameterization.yaml · Schema: curated_concept.yaml

Explore across X-Mech

Related computational & biological Mechs: Embedding Map Network Graph ProteinTraitsMech (ESM-2 pLM) DUFMech (Domain pLM) TraitMech CultureMech All 14 X-Mech Repositories