Home / Curated / Algorithms / CFL (Clustered Federated Learning via Cosine Similarity Bipartitioning)

CFL (Clustered Federated Learning via Cosine Similarity Bipartitioning)

Algorithms AIMech:CflClusteredFederatedLearningViaCosineSimilarityBipartitioning DRAFT

Aliases: CFL, Clustered Federated Learning via Cosine Similarity Bipartitioning, Recursive Cosine-Similarity Bipartitioning for Clustered Federated Learning

A recursive model-agnostic clustered federated learning algorithm (Sattler, Müller, & Samek 2021, Algorithm 1 & Algorithm 2) that detects client data incongruence whenever a cluster model theta^* approaches a stationary point of its aggregate empirical risk (0 <= ||sum_{i in c} (|D_i|/|D_c|) Delta theta_i|| < varepsilon_1) while individual client update norms remain large (max_{i in c} ||Delta theta_i|| > varepsilon_2 > 0, Eqs. 32-33), computes the pairwise cosine similarity matrix alpha_{i,j} = <Delta theta_i, Delta theta_j> / (||Delta theta_i|| ||Delta theta_j||) (Eq. 13), and recursively bipartitions the cluster into c_1 cup c_2 = c by minimizing the maximum cross-cluster similarity argmin_{c_1 cup c_2 = c} max_{i in c_1, j in c_2} alpha_{i,j} in O(|c|^3) time whenever the separation bound gamma_max < sqrt((1 - alpha_cross^max) / 2) (Eq. 39) holds.

Identity Notes & Concept Scope Distinct from IFCA (AIMech:IfcaIterativeFederatedClusteringAlgorithm, Ghosh et al. 2020), which requires specifying a fixed number of clusters k upfront and broadcasts all k cluster models to every client in each round for minimum-loss cluster assignment, whereas CFL requires no prior knowledge of k, communicates only a single cluster model per client per round, and recursively splits clusters top-down at stationary points via gradient cosine-similarity minimax bipartitioning.

Claims & Grounded Evidence

4 Claims

Mechanism #r414-cosine-similarity-separation-theorem-for-incongruent-distributions

Theorem 1 (Separation Theorem) and Corollary 1 prove that for m clients drawn from k incongruent data-generating distributions at a stationary point theta^* of the global objective with relative empirical gradient noise gamma_i = ||nabla R_{I(i)}(theta^*) - nabla r_i(theta^*)|| / ||nabla R_{I(i)}(theta^*)|| < 1, the minimum intra-cluster cosine similarity satisfies alpha_intra^min >= 1 - 2 gamma_max^2 while the minimax bipartition achieves cross-cluster similarity alpha_cross^max <= cos(pi / (k - 1)) H_{i^*,j^*} + sqrt(1 - cos^2(pi / (k - 1))) sqrt(1 - H_{i^*,j^*}^2) (where H_{i,j} = sqrt(1 - gamma_i^2) sqrt(1 - gamma_j^2) - gamma_i gamma_j), guaranteeing exact separation whenever the separation gap g(alpha) = alpha_intra^min - alpha_cross^max > 0.

supports Source: Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraints · Locator: Section IV-A Separation Theorem (Theorem 1, Corollary 1, Definition 2, Eqs. 17-31) & Appendix A (Lemmas 2-4, Proof of Theorem 5)

Mechanism #r414-stationary-point-incongruence-criterion-and-orthogonal-encryption-invariance

Splitting only when ||sum_{i in c} (|D_i|/|D_c|) Delta theta_i|| < varepsilon_1 and max_{i in c} ||Delta theta_i|| > varepsilon_2 prevents premature splitting on congruent clients while recursively building a parameter tree enabling instant warm-start re-clustering under distribution shifts; furthermore, because inner products and Euclidean norms are invariant under orthonormal transformations (<P x, P y> = <x, y>, ||P x|| = ||x||), CFL operates identically on client weight updates encrypted with a shared random permutation or orthogonal matrix P (Section V-B).

supports Source: Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraints · Locator: Section IV-B-C (Eqs. 32-44, Algorithm 2, Figure 2) & Section V Privacy Preservation (Eqs. 45-46)

Limitation #r414-stationary-point-wait-and-threshold-hyperparameter-limitation

CFL must wait for the current cluster model to approach a stationary point (||Delta theta_c|| < varepsilon_1) before triggering each bipartition step, and requires tuning the stationarity threshold varepsilon_1 and client incongruence threshold varepsilon_2.

supports Source: Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraints · Locator: Section IV-B (Eqs. 32-39, Table I)

Application #r414-label-swap-and-rotated-mnist-cifar10-and-agnews-clustering-application

Applied to multi-task federated learning under label-swapped and rotated MNIST, CIFAR-10, and AG-News benchmarks with CNN and LSTM architectures, recovering the ground-truth client clusters and outperforming single-model FL.

supports Source: Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraints · Locator: Section VI Experiments (Figures 3-8, Tables II-III)

⚠️ Curated Limitations & Negative Results

Requires waiting for the cluster model to reach approximate stationarity before each bipartition step.
Requires setting stationarity and incongruence norm thresholds (varepsilon_1, varepsilon_2).

Artificial Intelligence Ontology (AIO) Grounding

Mapped AIO Classes

AIO Target Class	Mapping Relation	Mapping Rationale
`AIO:FederatedLearning` Federated Learning	broad_match	CFL is a clustered multi-task optimization algorithm within Federated Learning.
`AIO:HierarchicalClustering` Hierarchical Clustering	related_match	CFL uses agglomerative minimax-linkage bipartitioning over pairwise client gradient cosine similarities to construct a hierarchical cluster tree.

Evidence Sources & Primary Literature

3 Sources

Source ID	Publication / Document	Type / Access	Reference & Checksum
`r414-primary-pdf`	Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraints Felix Sattler, Klaus-Robert Müller, Wojciech Samek	primary paper full_text	ID: https://doi.org/10.1109/TNNLS.2020.3015958 sha256:9645442ff909…
`r414-arxiv-abs`	Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraints (arXiv abstract) Felix Sattler, Klaus-Robert Müller, Wojciech Samek	primary paper abstract_only	ID: https://arxiv.org/abs/1910.01991 sha256:654f764040a4…
`r414-metadata-cr`	Clustered Federated Learning: Model-Agnostic Distributed Multi-Task Optimization under Privacy Constraints (Crossref metadata) Felix Sattler, Klaus-Robert Müller, Wojciech Samek	primary paper abstract_only	ID: https://doi.org/10.1109/TNNLS.2020.3015958 sha256:71e54573e3b8…

Ontology & Grounding Topology

Open in interactive network graph →

Curation Provenance

Curator: Antigravity

Method: ai_assisted

Model: Gemini

Created: 2026-10-19T08:17:00Z

Updated: 2026-10-19T08:17:00Z

Source File: data/curated/CflClusteredFederatedLearningViaCosineSimilarityBipartitioning.yaml · Schema: curated_concept.yaml

Explore across X-Mech

Related computational & biological Mechs: Embedding Map Network Graph ProteinTraitsMech (ESM-2 pLM) DUFMech (Domain pLM) TraitMech CultureMech All 14 X-Mech Repositories