Home / Curated / Algorithms / FedGKT (Group Knowledge Transfer for Federated Learning of Large CNNs at the Edge)

FedGKT (Group Knowledge Transfer for Federated Learning of Large CNNs at the Edge)

Algorithms AIMech:FedGktGroupKnowledgeTransferForEdgeFederatedLearning DRAFT

Aliases: FedGKT, Group Knowledge Transfer, Federated Group Knowledge Transfer

An alternating-minimization federated learning and split-knowledge-transfer algorithm (He, Annavaram, & Avestimehr 2020, Section 3, Equations 2-10, Algorithm 1) that trains compact edge CNNs W^(k) = (W_e^(k), W_c^(k)) on resource-constrained clients and a high-capacity server classifier W_s by exchanging extracted hidden feature maps H_i^(k) = f_e^(k)(X_i^(k); W_e^(k)) and soft logits z_c^(k),z_s, alternating between client updates minimizing l_c = l_CE(y_i, z_c^(k)) + D_KL(p_s || p_k) (Equations 7-8) and server updates minimizing l_s = sum_k sum_i [l_CE(y_i, z_s) + D_KL(p_k || p_s)] (Equations 5-6).

Identity Notes & Concept Scope Distinct from FedDF (AIMech:FedDfEnsembleDistillationForModelFusionInFederatedLearning, Lin et al. 2020), which transmits full client model weights to the server and performs ensemble distillation on an unlabeled server proxy dataset rather than offloading deep CNN layers to the server via uploaded client hidden feature maps and bidirectional ground-truth + KL distillation.

Claims & Grounded Evidence

3 Claims

Mechanism #r449-bidirectional-kl-distillation-and-feature-map-offloading-mechanism

Reformulating federated training as constrained alternating minimization (Equations 2-10, Algorithm 1) offloads deep residual blocks W_s to the server while each edge client k trains only a lightweight feature extractor W_e^(k) and compact head W_c^(k), reducing edge training FLOPs by 9x-17x and edge parameter memory by 54x-105x relative to on-device ResNet-56/ResNet-110 training while supports asynchronous client updates.

supports Source: Group Knowledge Transfer: Federated Learning of Large CNNs at the Edge · Locator: Section 3 Reformulation & Bidirectional Knowledge Transfer (Equations 4-10, Table 1), pp. 3-6

Limitation #r449-per-sample-feature-map-upload-and-label-sharing-limitation

FedGKT requires each client k to transmit intermediate feature maps H_i^(k) and corresponding labels y_i to the server for its local training examples, which scales communication volume with local dataset size |D_k| and exposes intermediate activations and labels unless combined with privacy mechanisms.

supports Source: Group Knowledge Transfer: Federated Learning of Large CNNs at the Edge · Locator: Section 3.2 Training Pipeline (Algorithm 1) & Section 5 Discussion, pp. 5-9

Application #r449-edge-cnn-cifar10-cifar100-and-cinic10-application

Applied to edge-server federated training of ResNet-56 and ResNet-110 classifiers with compact ResNet-8 edge extractors on CIFAR-10, CIFAR-100, and CINIC-10 under IID and non-IID partitions.

supports Source: Group Knowledge Transfer: Federated Learning of Large CNNs at the Edge · Locator: Section 4 Experiments (Tables 1-4, Figures 3-4), pp. 6-9

⚠️ Curated Limitations & Negative Results

Communication volume scales with the number of local training examples because per-sample intermediate feature maps H_i^(k) are transmitted.
Requires sharing labels y_i and intermediate activations H_i^(k) with the server.

Artificial Intelligence Ontology (AIO) Grounding

Mapped AIO Classes

AIO Target Class	Mapping Relation	Mapping Rationale
`AIO:FederatedLearning` Federated Learning	narrow_match	FedGKT is an edge-server federated learning method that combines split feature-map offloading with bidirectional KL knowledge transfer.

Evidence Sources & Primary Literature

3 Sources

Source ID	Publication / Document	Type / Access	Reference & Checksum
`r449-primary-pdf`	Group Knowledge Transfer: Federated Learning of Large CNNs at the Edge Chaoyang He, Murali Annavaram, Salman Avestimehr	primary paper full_text	ID: https://proceedings.neurips.cc/paper/2020/hash/a1d4c20b182ad7137ab3606f0e3fc8a4-Abstract.html sha256:b3357ffb1dc5…
`r449-arxiv-abs`	Group Knowledge Transfer: Federated Learning of Large CNNs at the Edge (arXiv abstract) Chaoyang He, Murali Annavaram, Salman Avestimehr	primary paper abstract_only	ID: https://arxiv.org/abs/2007.14513 sha256:47923e3a5617…
`r449-metadata-cr`	Group Knowledge Transfer: Federated Learning of Large CNNs at the Edge (Crossref metadata) Chaoyang He, Murali Annavaram, Salman Avestimehr	primary paper abstract_only	ID: https://proceedings.neurips.cc/paper/2020/hash/a1d4c20b182ad7137ab3606f0e3fc8a4-Abstract.html sha256:de2e046a8b32…

Ontology & Grounding Topology

Open in interactive network graph →

Curation Provenance

Curator: Antigravity

Method: ai_assisted

Model: Gemini

Created: 2026-10-19T08:52:00Z

Updated: 2026-10-19T08:52:00Z

Source File: data/curated/FedGktGroupKnowledgeTransferForEdgeFederatedLearning.yaml · Schema: curated_concept.yaml

Explore across X-Mech

Related computational & biological Mechs: Embedding Map Network Graph ProteinTraitsMech (ESM-2 pLM) DUFMech (Domain pLM) TraitMech CultureMech All 14 X-Mech Repositories