Invariant information clustering
Algorithms AIMech:InvariantInformationClustering DRAFT
Aliases: IIC
A neural clustering procedure that maximizes mutual information between discrete cluster-assignment variables for paired unlabeled samples, estimating their joint distribution from the network’s soft assignment probabilities.
Identity Notes & Concept Scope
Invariant information clustering (IIC) refers to Ji et al.’s paired-data discrete-information objective and neural learning procedure. It directly learns cluster assignments and differs from continuous embedding mutual-information estimators, k-means pseudo-label training and generic biclustering. No normalized identity match was found across all 442 pinned AIO labels/scoped synonyms and 195 prior native IDs/names/aliases. Ignored-inclusive, word-delimited searches of both full checkouts found nine prior discovery/review references in each plus six unrelated language-stemmer tokens in the main environment. An initial substring search also matched unrelated layer-name suffixes. Neighboring k-means, SwAV, Barlow Twins and VICReg mechanisms and exact AIO definitions/edges were inspected; shared components do not establish identity.
Claims & Grounded Evidence
1 Claim
Mechanism
#method
For n paired samples (x_i,x′_i), apply the same network ending in a C-class softmax to obtain probability vectors a_i and b_i. Define discrete labels z and z′ as conditionally independent given the input pair, with these probabilities. Form the batch joint estimate P=(1/n) sum_i a_i b_i^T and, for symmetric pair sampling, replace it by (P+P^T)/2. Obtain marginals r_c=sum_d P_cd and s_d=sum_c P_cd. Minimize −sum_(c,d) P_cd log[P_cd/(r_c s_d)], differentiating through both network outputs and their induced joint/marginals. Use the continuous zero-mass convention or a documented numerical stabilization. For images, pairs are generated by content-preserving transformations. An auxiliary overclustering head can learn more clusters while sharing the backbone with the main head; the reported training alternates the heads. For segmentation, align transformed prediction grids back to the original coordinates and average mutual information separately over local spatial displacements between patch assignments.
⚠️ Curated Limitations & Negative Results
- The information quantity is between modeled discrete assignment variables after averaging paired samples, not between the continuous probability vectors themselves. Finite-batch estimation and conditional modeling assumptions affect the objective.
- Constant assignments give zero mutual information, whereas a feasible balanced deterministic correspondence can give a larger value. This distinguishes the objective from pure agreement but does not prove arbitrary optimization escapes stationary collapsed states or recovers semantic classes.
- The entropy term favors cluster utilization, so chosen cluster count, class imbalance, distractors and augmentation semantics matter. High mutual information can preserve a nuisance factor shared by paired inputs; it is not proof of correct categories.
- The pinned author loss computes marginals before epsilon clamping; the paper’s compact Figure 4 clamps the joint first. These stabilized computations approximate the ideal normalized probability formula. The optional entropy coefficient differs from pure mutual information when it is not one.
- Geometric alignment is required for segmentation; same raw pixel coordinates need not denote corresponding content after a transform. Learned cluster indices have arbitrary order: evaluation maps them to reference classes using labels, and the paper’s semi-supervised modes must remain distinct from unlabeled training. No benchmark result was reproduced.
Artificial Intelligence Ontology (AIO) Grounding
Mapped AIO Classes
| AIO Target Class | Mapping Relation | Mapping Rationale |
|---|---|---|
AIO:SelfsupervisedLearning
Self-supervised Learning
|
related_match | This algorithm learns from self-generated targets or relationships between unlabeled inputs. The pinned AIO target is a MachineLearningTask; this is a related algorithm-to-task mapping, not equivalence or an is-a assertion. |
AIO:UnsupervisedClustering
Unsupervised Clustering
|
related_match | This procedure uses or directly learns unlabeled image cluster assignments. The pinned target is an unlabeled clustering task; this is a related algorithm-to-task association, not equivalence, a semantic class-label guarantee or an is-a assertion. |
Evidence Sources & Primary Literature
2 Sources| Source ID | Publication / Document | Type / Access | Reference & Checksum |
|---|---|---|---|
method-source
|
Xu Ji, João F. Henriques, Andrea Vedaldi
|
primary paper
full_text
|
ID: https://openaccess.thecvf.com/content_ICCV_2019/html/Ji_Invariant_Information_Clustering_for_Unsupervised_Image_Classification_and_Segmentation_ICCV_2019_paper.html
sha256:a1583c3265e1…
|
additional-1
|
xu-ji/IIC contributors
|
official documentation
documentation
|
ID: https://github.com/xu-ji/IIC/blob/b7602b743552c28a4af3de238bf2539a3af0338e/code/utils/cluster/IID_losses.py
sha256:292ccb2f8cfe…
|
Curation Provenance
Curator: Codex
Method: ai_assisted
Model: GPT-6
Created: 2026-10-11T08:15:29Z
Updated: 2026-10-11T08:15:29Z
Source File:
data/curated/InvariantInformationClustering.yaml