Home / Curated / Algorithms / 3PC Three Point Compressors for Communication-Efficient Distributed Training

3PC Three Point Compressors for Communication-Efficient Distributed Training

Algorithms AIMech:ThreePointCompressorsForCommunicationEfficientDistributedTraining DRAFT

Aliases: 3PC, Three Point Compressors, CLAG Compressed Lazy Aggregation

A unified class of parameterized compression mechanisms C_{h,y}: R^d -> R^d (Three Point Compressors, 3PC) conditioned on two state vectors h, y in R^d that satisfy E[||C_{h,y}(x) - x||^2] <= (1 - A)||h - y||^2 + B||x - y||^2 for constants A in (0, 1] and B >= 0, together with the 3PC distributed gradient descent algorithm (x^(t+1) = x^t - gamma * g^t, g_i^(t+1) = C_{g_i^t, grad f_i(x^t)}(grad f_i(x^(t+1)))) that unifies and hybridizes contractive error feedback (EF21), Lazily Aggregated Gradients (LAG), Compressed Lazy Aggregation (CLAG), and probabilistic shifting (3PCv1-3PCv5).

Identity Notes & Concept Scope Introduced by Peter Richtarik, Igor Sokolov, Ilyas Fatkhullin, Elnur Gasanov, Zhize Li, and Eduard Gorbunov (ICML 2022, PMLR Vol. 162, pp. 18596-18648; arXiv:2202.00998). Distinct from EF21 (Richtarik, Sokolov, & Fatkhullin, NeurIPS 2021; Round 380), which is a single special case of 3PC without lazy trigger thresholds; and distinct from standalone Lazily Aggregated Gradient (LAG, Chen et al., NeurIPS 2018), which uses a multi-step weighted history condition without contractive compression or the single-step 3PC Lyapunov inequality.

Claims & Grounded Evidence

4 Claims

Mechanism #threepc-three-point-contraction-and-clag-hybridization-mechanism

Given an alpha-contractive compressor C, 3PC instantiates EF21 via C_{h,y}^EF21(x) = h + C(x - h) with A = 1 - sqrt(1 - alpha) and B = (1 - alpha) / (1 - sqrt(1 - alpha)), instantiates single-step LAG via C_{h,y}^LAG(x) = x if ||x - h||^2 > zeta * ||x - y||^2 else h with A = 1 and B = zeta, and combines them into CLAG (Algorithm 4) via C_{h,y}^CLAG(x) = h + C(x - h) if ||x - h||^2 > zeta * ||x - y||^2 else h satisfying Definition 4.1 with A = 1 - sqrt(1 - alpha) and B = max{(1 - alpha)/(1 - sqrt(1 - alpha)), zeta}.

supports Source: 3PC: Three Point Compressors for Communication-Efficient Distributed Training and a Better Theory for Lazy Aggregation · Locator: Section 4.1-4.3 Equations (6)-(13), Table 1, & Appendix C Lemmas C.3, C.5, and C.8, pp. 4-6, 17-20

Scope #threepc-nonconvex-o-1-over-t-and-pl-linear-convergence-scope

For any 3PC compressor with parameters (A, B) applied to L_--smooth global and L_+-average-smooth local objectives with step size 0 < gamma <= 1 / (L_- + L_+ * sqrt(2*B/A)), Lemma 5.4 establishes the Distortion Lyapunov recursion E[G^(t+1)] <= (1 - A) E[G^t] + B * L_+^2 * E[||x^(t+1) - x^t||^2], Theorem 5.5 proves O(1/T) nonconvex stationarity without bounded gradient similarity, and Theorem 5.6 proves linear convergence under the Polyak-Lojasiewicz condition.

supports Source: 3PC: Three Point Compressors for Communication-Efficient Distributed Training and a Better Theory for Lazy Aggregation · Locator: Section 5 Assumptions 5.1-5.3, Lemma 5.4, Theorem 5.5, & Theorem 5.6, pp. 7-8

Limitation #threepc-worst-case-trigger-and-stochastic-minibatch-limitation

The deterministic worst-case iteration bound for LAG and CLAG depends on B = max{(1 - alpha)/(1 - sqrt(1 - alpha)), zeta}, which does not strictly improve the worst-case bound of EF21 when all workers trigger communication simultaneously, and stochastic extensions require either growing minibatches or PAGE/SPIDER variance reduction.

supports Source: 3PC: Three Point Compressors for Communication-Efficient Distributed Training and a Better Theory for Lazy Aggregation · Locator: Section 4.2-4.3 Remarks & Appendix G Stochastic Extensions, pp. 5-6, 37-45

Application #threepc-clag-nonconvex-logistic-regression-and-resnet18-cifar10-application

Applied to distributed nonconvex logistic regression across LIBSVM datasets (phishing, w8a, a9a, mushrooms) and distributed training of ResNet-18 on CIFAR-10, where CLAG and 3PCv1-3PCv5 reduce total communicated bits compared to standalone EF21 and LAG.

supports Source: 3PC: Three Point Compressors for Communication-Efficient Distributed Training and a Better Theory for Lazy Aggregation · Locator: Section 6 Experiments & Appendix B Additional Experiments, pp. 8-9, 13-16

⚠️ Curated Limitations & Negative Results

Requires each worker to store previous gradient and iterate states to evaluate the three-point operator.
Lazy upload skipping gains depend on inter-worker smoothness heterogeneity and the trigger parameter zeta.

Artificial Intelligence Ontology (AIO) Grounding

Mapped AIO Classes

AIO Target Class	Mapping Relation	Mapping Rationale
`AIO:FederatedLearning` Federated Learning	related_match	AIO:FederatedLearning covers distributed model training across networked nodes, whereas 3PC is a parameterized three-point compression and lazy-aggregation framework for communication-efficient distributed optimization.

Evidence Sources & Primary Literature

3 Sources

Source ID	Publication / Document	Type / Access	Reference & Checksum
`r559-primary-pdf`	3PC: Three Point Compressors for Communication-Efficient Distributed Training and a Better Theory for Lazy Aggregation Peter Richtarik, Igor Sokolov, Ilyas Fatkhullin, Elnur Gasanov, Zhize Li, Eduard Gorbunov	primary paper full_text	ID: https://openalex.org/W4221138265 sha256:7e70b8c4886b…
`r559-arxiv-abs`	3PC: Three Point Compressors for Communication-Efficient Distributed Training and a Better Theory for Lazy Aggregation (arXiv abstract) Peter Richtarik, Igor Sokolov, Ilyas Fatkhullin, Elnur Gasanov, Zhize Li, Eduard Gorbunov	primary paper abstract_only	ID: https://arxiv.org/abs/2202.00998 sha256:b83055098acc…
`r559-metadata-cr`	3PC: Three Point Compressors for Communication-Efficient Distributed Training and a Better Theory for Lazy Aggregation (Crossref metadata) Peter Richtarik, Igor Sokolov, Ilyas Fatkhullin, Elnur Gasanov, Zhize Li, Eduard Gorbunov	primary paper abstract_only	ID: https://openalex.org/W4221138265 sha256:f61882478147…

Ontology & Grounding Topology

Open in interactive network graph →

Curation Provenance

Curator: Antigravity

Method: ai_assisted

Model: Gemini

Created: 2026-10-19T10:42:00Z

Updated: 2026-10-19T10:42:00Z

Source File: data/curated/ThreePointCompressorsForCommunicationEfficientDistributedTraining.yaml · Schema: curated_concept.yaml

Explore across X-Mech

Related computational & biological Mechs: Embedding Map Network Graph ProteinTraitsMech (ESM-2 pLM) DUFMech (Domain pLM) TraitMech CultureMech All 14 X-Mech Repositories