AIMech AI

Sparse Expectation Maximization

Algorithms AIMech:SparseExpectationMaximization DRAFT
Aliases: Sparse EM
An EM variant that updates relative latent-state probabilities within selected plausible sets while preserving their total mass and all outside probabilities, and fits parameters using the resulting full distributions with occasional complete posterior refreshes.
Identity Notes & Concept Scope Sparse denotes selective posterior work in the Neal-Hinton construction, not sparse model weights, pruning or setting excluded states to zero. It differs from incremental EM’s selection of observations and can be combined with that method. It is also distinct from restricting a variational posterior to independent chain factors or permanently imposing a truncated support. No normalized identity matches among 442 pinned and 476 maintained records, including acronym probes. GEM is the exact-current-E/Q-improvement framework, IEM refreshes cached independent cases, sparse EM preserves frozen outside mass, and DAEM is the warm-started cooling-to-beta1 procedure. Existing TEM has the same finite-state fixed-beta update, but its broader update framework and held-out inverse-annealing realization do not define that outer cooling procedure. ECM, AECM and SAGE specialize parameter/augmentation updates, not these case or probability-subset operations. Structured factorial-HMM variational inference imposes a different family restriction. Names and incidental frontier references do not establish identity; qualified GEM/IEM aliases avoid unrelated uses. The absence assessment is bounded to this snapshot and corpus. Identity-search counts: 253 matched lines in the worktree and 260 in main; all 98 distinct matching-text windows were read, with ignored and hidden files included.

Claims & Grounded Evidence

3 Claims
Mechanism #frozen-mass-update
For a case with cached q and plausible set S, freeze every q_z outside S and the total Q=sum_(z in S)q_z. At the current theta compute r_z=p(z|x,theta)/sum_(u in S)p(u|x,theta) within S and set q_z=Q*r_z there. The restricted normalizer must be positive and finite. The M-step uses the full expected complete log likelihood, including outside contributions; cached sufficient statistics can avoid recomputing unchanged terms.
Scope #refresh-and-objectives
Occasionally recompute the full posterior to refresh outside probabilities, plausible sets and their total mass. The source suggests top-count or cumulative-mass sets as heuristics, without one universal optimal threshold or refresh frequency. Exact restricted E and admissible M updates make the entropy-augmented F nondecreasing, but the source explicitly permits observed likelihood to decrease. Zeroing outside mass and renormalizing retained states to one implements a different approximation.
Limitation #conditional-efficiency
Savings require a posterior concentrated on a manageable plausible set and a model for which unchanged contributions can be reused. Full refreshes and model-specific M-steps can dominate cost. The source’s stability wording is not a general proof of bounded iterates, parameter convergence or global optimization; admissible support, repeated refreshes and optimization conditions still need analysis.

⚠️ Curated Limitations & Negative Results

Artificial Intelligence Ontology (AIO) Grounding

Mapped AIO Classes
AIO Target Class Mapping Relation Mapping Rationale
AIO:MachineLearningTask
Machine Learning Task
related_match This procedure fits latent-model parameters from observed data. The inspected AIO target denotes a learning process/task; this algorithm-to-task relationship asserts neither equivalence nor an is-a relation.

Evidence Sources & Primary Literature

1 Source
Source ID Publication / Document Type / Access Reference & Checksum
neal-source
Radford M. Neal, Geoffrey E. Hinton
primary paper
full_text
ID: https://www.cs.toronto.edu/~hinton/absps/emk.pdf
sha256:6b07cbc3c578…

Curation Provenance

Curator: Codex
Method: ai_assisted
Model: GPT-6
Created: 2026-10-12T04:39:47Z
Updated: 2026-10-12T04:39:47Z
Source File: data/curated/SparseExpectationMaximization.yaml