Home / Curated / Algorithms / FedProto (Federated Prototype Learning across Heterogeneous Clients)

FedProto (Federated Prototype Learning across Heterogeneous Clients)

Algorithms AIMech:FedProtoFederatedPrototypeLearningAcrossHeterogeneousClients DRAFT

Aliases: FedProto, Federated Prototype Learning, FedProto-mh

A gradient-free and parameter-free heterogeneous federated learning framework (Tan, Long, Liu, Zhou, Lu, Jiang, & Zhang 2022, Sections Problem Setting & Methodology, Equations 2-8, Algorithm 1, Theorems 1-2) in which each client i with potentially heterogeneous model architecture F_i(phi_i, nu_i) = g_i(nu_i) o f_i(phi_i) computes local class prototypes C_i^{(j)} = (1 / |D_{i,j}|) sum_{(x,y) in D_{i,j}} f_i(phi_i; x) in a shared d_c-dimensional embedding space (Equation 3), transmits only its local prototype set C_i to the central server, receives back global class prototypes C_bar^{(j)} aggregated across the subset of clients N_j holding class j via sample-weighted averaging (Equation 6), and updates its local model parameters omega_i = (phi_i, nu_i) by minimizing the prototype-regularized local objective L(D_i, omega_i) = L_S(F_i(omega_i; x), y) + lambda sum_j d(C_i^{(j)}, C_bar_i^{(j)}) (Equations 7-8).

Identity Notes & Concept Scope Distinct from FedPer (AIMech:FedPerFederatedLearningWithPersonalizationLayers, Arivazhagan et al. 2019) and FedRep (AIMech:FedRepExploitingSharedRepresentationsForPersonalizedFederatedLearning, Collins et al. 2021), which exchange and average homogeneous base-layer weight tensors in parameter space; and distinct from FedGen (AIMech:FedGenDataFreeFederatedKnowledgeDistillationViaServerLatentGenerator, Zhu, Hong, & Zhou 2021), which trains a conditional generator on the server from uploaded classifier heads rather than aggregating mean class feature prototypes C_bar^{(j)} across architecturally heterogeneous encoders.

Claims & Grounded Evidence

4 Claims

Mechanism #r474-heterogeneous-architecture-prototype-aggregation-mechanism

By exchanging only d_c-dimensional class prototype vectors C_i^{(j)} rather than gradients or model weights omega_i, FedProto allows clients with distinct neural architectures F_i (e.g., different convolutional channel widths or strides in FedProto-mh) and partially overlapping class subsets to align their embedding spaces around global class centroids C_bar^{(j)} while reducing communicated parameters by orders of magnitude (Tan et al. 2022, Methodology & Experiments, Equations 3-8, Table 1).

supports Source: FedProto: Federated Prototype Learning across Heterogeneous Clients · Locator: Methodology (Equations 3-8, Algorithm 1), pp. 3-4, and Experiments (Table 1), pp. 6-7

Mechanism #r474-nonconvex-one-round-deviation-and-convergence-rate-mechanism

Under L_1-Lipschitz smoothness of local objectives, bounded stochastic gradient variance sigma^2 and norm G, and L_2-Lipschitz continuity of local embedding functions f_i(phi_i) (Assumptions 1-4), Theorem 1 bounds the one-round expected loss change by E[L_{(t+1)E+1/2}] <= L_{tE+1/2} - (eta - L_1 eta^2 / 2) sum_{e=1/2}^{E-1} ||nabla L_{tE+e}||_2^2 + (L_1 E eta^2 / 2) sigma^2 + lambda L_2 eta E G, and Theorem 2 establishes non-convex convergence to an epsilon-stationary point in T = 2 Delta / (E epsilon (2 eta - L_1 eta^2) - E eta (L_1 eta sigma^2 + 2 lambda L_2 G)) rounds (Tan et al. 2022, Convergence Analysis, Equations 9-19).

supports Source: FedProto: Federated Prototype Learning across Heterogeneous Clients · Locator: Convergence Analysis (Assumptions 1-4, Theorems 1-2, Corollary 1, Equations 9-19), pp. 4-5, and Supplement pp. 12-16

Application #r474-few-shot-non-iid-and-model-heterogeneous-fl-application

Applied to n-way k-shot statistically and architecturally heterogeneous federated classification on MNIST, FEMNIST, and CIFAR-10 (Tan et al. 2022, Experiments, Tables 1-4, Figures 2-4).

supports Source: FedProto: Federated Prototype Learning across Heterogeneous Clients · Locator: Experiments (Tables 1-4, Figures 2-4), pp. 6-7 & 10-11

Limitation #r474-shared-embedding-dimension-and-lambda-tuning-limitation

Requires all heterogeneous client encoders f_i to map into a common prototype dimension d_c and requires tuning the prototype regularization weight lambda (Corollary 1 requires lambda < ||nabla L_{tE+1/2}||_2^2 / (L_2 E G) to prevent server prototype aggregation from increasing the local objective) (Tan et al. 2022, Corollary 1, Figure 4, & Supplement p. 12).

supports Source: FedProto: Federated Prototype Learning across Heterogeneous Clients · Locator: Corollary 1 (Equation 17), p. 5, Figure 4, p. 7, and Additional Notation, p. 12

⚠️ Curated Limitations & Negative Results

Requires all clients' representation layers f_i to output embeddings of identical dimension d_c so class prototypes can be averaged on the server.
Does not produce a single standalone classifier on the server without pairing global prototypes with a pre-trained or client-side feature extractor.

Artificial Intelligence Ontology (AIO) Grounding

Mapped AIO Classes

AIO Target Class	Mapping Relation	Mapping Rationale
`AIO:FederatedLearning` Federated Learning	broad_match	FedProto is a prototype-aggregation federated learning algorithm within the broader FederatedLearning category.

Evidence Sources & Primary Literature

3 Sources

Source ID	Publication / Document	Type / Access	Reference & Checksum
`r474-primary-pdf`	FedProto: Federated Prototype Learning across Heterogeneous Clients Yue Tan, Guodong Long, Lu Liu, Tianyi Zhou, Qinghua Lu, Jing Jiang, Chengqi Zhang	primary paper full_text	ID: https://arxiv.org/abs/2105.00243 sha256:7917a3ebf569…
`r474-arxiv-abs`	FedProto: Federated Prototype Learning across Heterogeneous Clients (arXiv abstract) Yue Tan, Guodong Long, Lu Liu, Tianyi Zhou, Qinghua Lu, Jing Jiang, Chengqi Zhang	primary paper abstract_only	ID: https://arxiv.org/abs/2105.00243 sha256:212623fe822d…
`r474-metadata-cr`	FedProto: Federated Prototype Learning across Heterogeneous Clients (Crossref metadata) Yue Tan, Guodong Long, Lu Liu, Tianyi Zhou, Qinghua Lu, Jing Jiang, Chengqi Zhang	primary paper abstract_only	ID: https://arxiv.org/abs/2105.00243 sha256:80b980e78595…

Ontology & Grounding Topology

Open in interactive network graph →

Curation Provenance

Curator: Antigravity

Method: ai_assisted

Model: Gemini

Created: 2026-10-19T09:17:00Z

Updated: 2026-10-19T09:17:00Z

Source File: data/curated/FedProtoFederatedPrototypeLearningAcrossHeterogeneousClients.yaml · Schema: curated_concept.yaml

Explore across X-Mech

Related computational & biological Mechs: Embedding Map Network Graph ProteinTraitsMech (ESM-2 pLM) DUFMech (Domain pLM) TraitMech CultureMech All 14 X-Mech Repositories