Incremental Expectation Maximization
Algorithms AIMech:IncrementalExpectationMaximization DRAFT
Aliases: Incremental EM, IEM (incremental expectation maximization)
An EM variant for independent observed cases that refreshes selected cached case posteriors at the current parameters and then fits parameters using the complete collection of refreshed and retained case expectations.
Identity Notes & Concept Scope
This record denotes the Neal-Hinton exact-cache fixed-data construction. It revisits a fixed data set and subtracts each selected case’s old contribution before adding its new one. It is distinct from streaming observation arrival, exponentially decayed online statistics, decreasing-step SAEM, arbitrary Gibbs updates of dependent latent coordinates and fixed-current-E GEM. Small batches of cases are a scheduling realization rather than a separate identity. Bare IEM is not asserted as an unambiguous alias. No normalized identity matches among 442 pinned and 476 maintained records, including acronym probes. GEM is the exact-current-E/Q-improvement framework, IEM refreshes cached independent cases, sparse EM preserves frozen outside mass, and DAEM is the warm-started cooling-to-beta1 procedure. Existing TEM has the same finite-state fixed-beta update, but its broader update framework and held-out inverse-annealing realization do not define that outer cooling procedure. ECM, AECM and SAGE specialize parameter/augmentation updates, not these case or probability-subset operations. Structured factorial-HMM variational inference imposes a different family restriction. Names and incidental frontier references do not establish identity; qualified GEM/IEM aliases avoid unrelated uses. The absence assessment is bounded to this snapshot and corpus. Identity-search counts: 253 matched lines in the worktree and 260 in main; all 98 distinct matching-text windows were read, with ignored and hidden files included.
Claims & Grounded Evidence
4 Claims
Mechanism
#case-cache
For p(X,Z|theta)=product_i p(x_i,z_i|theta), retain a distribution q_i or its sufficient statistics for every case. Select a case or batch, replace each selected q_i with p(z_i|x_i,theta_current), and leave all other q_i unchanged. Maximize sum_i E_q_i log p(x_i,z_i|theta) over admissible theta using every cached case. With additive statistics, S_new=S_old-s_i_old+s_i_new; forgetting the subtraction or fitting only the selected observation changes the method.
Scope
#bound-not-likelihood
Exact selected-posterior updates and admissible full-cache M-steps make F=sum_i E_q_i log p(x_i,z_i|theta)+sum_i H(q_i) nondecreasing. F equals observed log likelihood only when every cached distribution is the current posterior. The published correction explicitly shows incremental EM need not be GEM and displays likelihood decreases at early and later updates; ordinary observed-likelihood monotonicity cannot be inherited from ordinary EM.
Limitation
#convergence-and-cost
The published stationary-point result is conditional on the underlying standard-EM regularity and initialization assumptions plus uniqueness of the maximizer of the full cached sum for every tuple of case anchors and full support of the complete-data family. The comment cites an external proof; uniqueness and support alone are not asserted sufficient here. Storage for per-case caches and rounding drift also matter. Fewer passes are not necessarily less runtime because many more M-steps can be required.
Scope
#decay-boundary
Replacing the cache total by gamma*S_old+s_i_new with fixed 0<gamma<1 is the separate decayed-statistics comparator. It does not maintain the exact sum of one current cached contribution per fixed observation and is not certified to optimize the same fixed-data likelihood. The source also describes small batches and occasional total recomputation to manage overhead and roundoff.
⚠️ Curated Limitations & Negative Results
- Independent complete-data cases are part of this formulation. A partial update to one coordinate of a coupled latent vector is not justified by the independent-case formula.
- Fair revisiting, admissible M-step solutions, support and compactness/regularity conditions require a problem-specific analysis. No unconditional parameter convergence, local/global maximum or universal speedup is claimed.
- The original1998HMM article and referenced convergence report were not fully inspected. A finite curator cache-state example shows F increase with likelihood decrease; it does not reproduce the source’s complete training trajectory.
- No author software or historical empirical study was reproduced. Finite curator calculations illustrate stated objective and schedule boundaries; they are not a general convergence proof or an empirical speed comparison.
Artificial Intelligence Ontology (AIO) Grounding
Mapped AIO Classes
| AIO Target Class | Mapping Relation | Mapping Rationale |
|---|---|---|
AIO:MachineLearningTask
Machine Learning Task
|
related_match | This procedure fits latent-model parameters from observed data. The inspected AIO target denotes a learning process/task; this algorithm-to-task relationship asserts neither equivalence nor an is-a relation. |
Evidence Sources & Primary Literature
2 Sources| Source ID | Publication / Document | Type / Access | Reference & Checksum |
|---|---|---|---|
neal-source
|
Radford M. Neal, Geoffrey E. Hinton
|
primary paper
full_text
|
ID: https://www.cs.toronto.edu/~hinton/absps/emk.pdf
sha256:6b07cbc3c578…
|
correction-source
|
William Byrne, Asela Gunawardana
|
primary paper
full_text
|
ID: https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/byrne00__commen_train_algor_using_increm.pdf
sha256:3b92702a985c…
|
Curation Provenance
Curator: Codex
Method: ai_assisted
Model: GPT-6
Created: 2026-10-12T04:39:47Z
Updated: 2026-10-12T04:47:50Z
Source File:
data/curated/IncrementalExpectationMaximization.yaml