Home / Curated / Algorithms / PruneFL: Adaptive and Distributed Parameter Pruning for Federated Learning

PruneFL: Adaptive and Distributed Parameter Pruning for Federated Learning

Algorithms AIMech:PruneFlAdaptiveDistributedParameterPruningForFederatedLearning DRAFT

Aliases: PruneFL, Adaptive and Distributed Parameter Pruning for Federated Learning

An adaptive distributed parameter pruning and reconfiguration framework for federated learning (PruneFL; Jiang, Wang, Valls, Ko, Lee, Leung, and Tassiulas, IEEE TNNLS 2023) that executes initial iterative pruning on a single selected client prior to federated training and subsequently reconfigures (prunes and regrows) the shared sparse parameter set M = A union P_bar every R_r rounds during multi-client federated training (Algorithm 1). At each reconfiguration step, PruneFL models the single-step empirical risk reduction as Delta(M) approx sum_{j in M} g_j^2 (Eq. 6) and the single-round execution time as T(M) = c + sum_{j in M} t_j, and maximizes the risk-reduction rate Gamma(A union P_bar) = Delta(A union P_bar) / T(A union P_bar) (Eq. 7) over prunable indices P via a greedy O(|P| log |P|) sorting algorithm (Algorithm 2) that orders j in P in non-increasing order of g_j^2 / t_j and includes j while g_j^2 / t_j >= Gamma(A union P_bar).

Identity Notes & Concept Scope Distinct from personalized sparse federated learning methods such as DisPFL (AIMech:DisPflDecentralizedSparseTrainingForPersonalizedFederatedLearning; Dai et al., ICML 2022), which maintain client-specific fixed-density masks over a decentralized peer-to-peer graph rather than dynamically resizing a shared global model mask M_k via an exact greedy optimizer of the risk-reduction-per-unit-time ratio Gamma(M). Distinct from static post-training or lottery-ticket initialization pruning (Frankle & Carbin, 2019), which fixes a predetermined target sparsity ratio and requires rewinding weights to initialization rather than adaptively shrinking and regrowing the active parameter set during ongoing federated optimization.

Claims & Grounded Evidence

4 Claims

Mechanism #prunefl-greedy-risk-reduction-per-time-reconfiguration-optimality-mechanism

Given non-prunable parameter set A and prunable candidate set P with squared gradient components g_j^2 and per-round execution time costs t_j > 0, Theorem 1 (p. 5; proof via Lemmas 1-2 in Appendix B-1, pp. 14-15) proves that Algorithm 2—which sorts j in P in non-increasing order of g_j^2 / t_j and greedily adds j to P_bar while g_j^2 / t_j >= Gamma(A union P_bar)—finds a global maximizer of Gamma(A union P_bar) = (Delta(A) + sum_{j in P_bar} g_j^2) / (c + T_A + sum_{j in P_bar} t_j) in O(|P| log |P|) time.

supports Source: Model Pruning Enables Efficient Federated Learning on Edge Devices · Locator: Section IV-A to IV-C, Eqs. (6)-(7), Algorithm 2, Theorem 1, pp. 4-5; Appendix B-1, Lemmas 1-2, pp. 14-15

Scope #prunefl-masked-gradient-stationary-point-convergence-rate-scope

Under Assumptions 1-4 (L-smoothness, bounded mini-batch variance sigma^2, bounded full-gradient second moment G^2, and non-expansiveness ||w - w odot m||^2 <= ||w - w*||^2 of mask projection), Theorem 2 (p. 6; proof in Appendix B-2, pp. 15-19) proves that PruneFL with step size eta = sqrt(N / K) achieves (1 / K) sum_{k=0}^{K-1} E[||nabla F(w'(k)) odot m_{w'}(k)||^2] <= 2(E[F(w'(0))] - F*) / sqrt(N K) + (4 tau^2 L^2 G^2 / K) + (2 L sigma^2 / sqrt(N K)) = O(1 / sqrt(N K)) + O(1 / K).

supports Source: Model Pruning Enables Efficient Federated Learning on Edge Devices · Locator: Section IV-D Convergence Analysis, Assumptions 1-4, Theorem 2, p. 6; Appendix B-2, Lemma 3 and Proof of Theorem 2, pp. 15-19

Limitation #prunefl-profiling-calibration-and-dense-gradient-accumulation-limitation

PruneFL requires offline device profiling to estimate per-parameter round-time coefficients t_j and overhead constant c via linear regression, relies on a trusted client's local dataset for Stage-1 initial pruning, and requires clients to accumulate exponential moving averages of full-dimensional squared gradients g_j^2 across reconfiguration intervals (Section III-A to III-B, pp. 3-4; Section V-A, pp. 6-7).

supports Source: Model Pruning Enables Efficient Federated Learning on Edge Devices · Locator: Section III-A Initial Pruning at a Selected Client, Section III-B, pp. 3-4; Section V-A Time Profiling, Fig. 4, pp. 6-7; Appendix A, pp. 12-13

Application #prunefl-raspberry-pi-edge-federated-training-application

Applied to edge-device federated training on Raspberry Pi 4 and Android devices across FEMNIST, CIFAR-10, ImageNet-100, and CelebA using Conv-2, VGG-11, ResNet-18, and MobileNetV2.

supports Source: Model Pruning Enables Efficient Federated Learning on Edge Devices · Locator: Section V Experimental Results, Figs. 4-11, Tables I-II, pp. 6-11

⚠️ Curated Limitations & Negative Results

Requires device-specific linear-regression time profiling of t_j and c to calibrate the denominator of Gamma(A union P_bar) (Section V-A, Fig. 4, pp. 6-7).
Stage-1 initial pruning relies on a single selected client's local data distribution before federated training begins, which can temporarily bias the initial mask under severe non-IID data until Stage-2 server reconfigurations regrow missing parameters (Section III-A, p. 3; Section V-E, p. 9).
Unstructured weight pruning requires sparse tensor algebra libraries (such as custom sparse-dense matrix multiplication kernels) to achieve actual computation and memory savings on edge CPUs (Section V-A, p. 6; Appendix A, pp. 12-13).

Artificial Intelligence Ontology (AIO) Grounding

Mapped AIO Classes

AIO Target Class	Mapping Relation	Mapping Rationale
`AIO:FederatedLearning` Federated Learning	broad_match	PruneFL is an adaptive distributed parameter pruning algorithm for federated learning on resource-constrained edge devices.
`AIO:SparseLearning` Sparse Learning	related_match	PruneFL dynamically optimizes a sparse parameter mask to maximize empirical risk reduction per unit round time.

Evidence Sources & Primary Literature

3 Sources

Source ID	Publication / Document	Type / Access	Reference & Checksum
`r508-primary-pdf`	Model Pruning Enables Efficient Federated Learning on Edge Devices Yuang Jiang, Shiqiang Wang, Victor Valls, Bong Jun Ko, Wei-Han Lee, Kin K. Leung, Leandros Tassiulas	primary paper full_text	ID: https://doi.org/10.1109/tnnls.2022.3166101 sha256:90bb3e9d9991…
`r508-arxiv-abs`	Model Pruning Enables Efficient Federated Learning on Edge Devices (arXiv abstract) Yuang Jiang, Shiqiang Wang, Victor Valls, Bong Jun Ko, Wei-Han Lee, Kin K. Leung, Leandros Tassiulas	primary paper abstract_only	ID: https://arxiv.org/abs/1909.12326 sha256:b631d6ac891b…
`r508-metadata-cr`	Model Pruning Enables Efficient Federated Learning on Edge Devices (Crossref metadata) Yuang Jiang, Shiqiang Wang, Victor Valls, Bong Jun Ko, Wei-Han Lee, Kin K. Leung, Leandros Tassiulas	primary paper abstract_only	ID: https://doi.org/10.1109/tnnls.2022.3166101 sha256:af3579897425…

Ontology & Grounding Topology

Open in interactive network graph →

Curation Provenance

Curator: Antigravity

Method: ai_assisted

Model: Gemini

Created: 2026-10-19T09:51:00Z

Updated: 2026-10-19T09:51:00Z

Source File: data/curated/PruneFlAdaptiveDistributedParameterPruningForFederatedLearning.yaml · Schema: curated_concept.yaml

Explore across X-Mech

Related computational & biological Mechs: Embedding Map Network Graph ProteinTraitsMech (ESM-2 pLM) DUFMech (Domain pLM) TraitMech CultureMech All 14 X-Mech Repositories