Sparse Expectation Maximization
Algorithms AIMech:SparseExpectationMaximization DRAFT
Aliases: Sparse EM
An EM variant that updates relative latent-state probabilities within selected plausible sets while preserving their total mass and all outside probabilities, and fits parameters using the resulting full distributions with occasional complete posterior refreshes.
Identity Notes & Concept Scope
Sparse denotes selective posterior work in the Neal-Hinton construction, not sparse model weights, pruning or setting excluded states to zero. It differs from incremental EM’s selection of observations and can be combined with that method. It is also distinct from restricting a variational posterior to independent chain factors or permanently imposing a truncated support. No normalized identity matches among 442 pinned and 476 maintained records, including acronym probes. GEM is the exact-current-E/Q-improvement framework, IEM refreshes cached independent cases, sparse EM preserves frozen outside mass, and DAEM is the warm-started cooling-to-beta1 procedure. Existing TEM has the same finite-state fixed-beta update, but its broader update framework and held-out inverse-annealing realization do not define that outer cooling procedure. ECM, AECM and SAGE specialize parameter/augmentation updates, not these case or probability-subset operations. Structured factorial-HMM variational inference imposes a different family restriction. Names and incidental frontier references do not establish identity; qualified GEM/IEM aliases avoid unrelated uses. The absence assessment is bounded to this snapshot and corpus. Identity-search counts: 253 matched lines in the worktree and 260 in main; all 98 distinct matching-text windows were read, with ignored and hidden files included.
Claims & Grounded Evidence
3 Claims
Mechanism
#frozen-mass-update
For a case with cached q and plausible set S, freeze every q_z outside S and the total Q=sum_(z in S)q_z. At the current theta compute r_z=p(z|x,theta)/sum_(u in S)p(u|x,theta) within S and set q_z=Q*r_z there. The restricted normalizer must be positive and finite. The M-step uses the full expected complete log likelihood, including outside contributions; cached sufficient statistics can avoid recomputing unchanged terms.
Scope
#refresh-and-objectives
Occasionally recompute the full posterior to refresh outside probabilities, plausible sets and their total mass. The source suggests top-count or cumulative-mass sets as heuristics, without one universal optimal threshold or refresh frequency. Exact restricted E and admissible M updates make the entropy-augmented F nondecreasing, but the source explicitly permits observed likelihood to decrease. Zeroing outside mass and renormalizing retained states to one implements a different approximation.
Limitation
#conditional-efficiency
Savings require a posterior concentrated on a manageable plausible set and a model for which unchanged contributions can be reused. Full refreshes and model-specific M-steps can dominate cost. The source’s stability wording is not a general proof of bounded iterates, parameter convergence or global optimization; admissible support, repeated refreshes and optimization conditions still need analysis.
⚠️ Curated Limitations & Negative Results
- An outside probability may remain nonzero. The saved finite example preserves outside mass and increases F while decreasing likelihood from an admissible cached state; reachability from a particular full-posterior initialization is not claimed.
- The finite-state exposition does not automatically establish an arbitrary continuous-latent or dependent-factor sparse-update algorithm. A changing set requires explicit full-distribution bookkeeping.
- This construction is not a guarantee of exact posterior recovery, a fixed truncation tolerance, universal acceleration or neural-network sparsity.
- No author software or historical empirical study was reproduced. Finite curator calculations illustrate stated objective and schedule boundaries; they are not a general convergence proof or an empirical speed comparison.
Artificial Intelligence Ontology (AIO) Grounding
Mapped AIO Classes
| AIO Target Class | Mapping Relation | Mapping Rationale |
|---|---|---|
AIO:MachineLearningTask
Machine Learning Task
|
related_match | This procedure fits latent-model parameters from observed data. The inspected AIO target denotes a learning process/task; this algorithm-to-task relationship asserts neither equivalence nor an is-a relation. |
Evidence Sources & Primary Literature
1 Source| Source ID | Publication / Document | Type / Access | Reference & Checksum |
|---|---|---|---|
neal-source
|
Radford M. Neal, Geoffrey E. Hinton
|
primary paper
full_text
|
ID: https://www.cs.toronto.edu/~hinton/absps/emk.pdf
sha256:6b07cbc3c578…
|
Curation Provenance
Curator: Codex
Method: ai_assisted
Model: GPT-6
Created: 2026-10-12T04:39:47Z
Updated: 2026-10-12T04:39:47Z
Source File:
data/curated/SparseExpectationMaximization.yaml