Home / Curated / Algorithms / Generalized Expectation Maximization

Generalized Expectation Maximization

Algorithms AIMech:GeneralizedExpectationMaximization DRAFT

Aliases: Generalized EM, GEM (generalized expectation maximization)

A latent-variable likelihood-fitting framework that computes the expected complete-data log likelihood under the current exact posterior and replaces full maximization with a parameter update that does not decrease that same expected objective.

Identity Notes & Concept Scope GEM generalizes the M-step requirement of ordinary EM; it does not license an arbitrary approximate E-step. Exact EM and suitable fixed-Q ECM updates are special cases. Cached-case incremental EM, Monte Carlo approximations and ECME actual-likelihood steps do not automatically satisfy the current-Q GEM inequality. The source-defined framework is distinct from any particular gradient, coordinate or space-alternating implementation. Bare GEM is not asserted as a globally unique alias. No normalized identity matches among 442 pinned and 476 maintained records, including acronym probes. GEM is the exact-current-E/Q-improvement framework, IEM refreshes cached independent cases, sparse EM preserves frozen outside mass, and DAEM is the warm-started cooling-to-beta1 procedure. Existing TEM has the same finite-state fixed-beta update, but its broader update framework and held-out inverse-annealing realization do not define that outer cooling procedure. ECM, AECM and SAGE specialize parameter/augmentation updates, not these case or probability-subset operations. Structured factorial-HMM variational inference imposes a different family restriction. Names and incidental frontier references do not establish identity; qualified GEM/IEM aliases avoid unrelated uses. The absence assessment is bounded to this snapshot and corpus. Identity-search counts: 253 matched lines in the worktree and 260 in main; all 98 distinct matching-text windows were read, with ignored and hidden files included.

Claims & Grounded Evidence

3 Claims

Mechanism #exact-anchor-partial-maximization

At theta_t form Q(theta|theta_t)=E_[Z|X,theta_t] log p(X,Z|theta), with the conditional law fixed throughout the parameter update. Choose an admissible theta_(t+1) satisfying Q(theta_(t+1)|theta_t)>=Q(theta_t|theta_t), then refresh the expectation. An exact argmax is allowed but not required; a numerical step needs an actual improvement check rather than the name of an optimizer.

supports Source: Comments on “Efficient Training Algorithms for HMM’s Using Incremental Estimation” · Locator: SectionII, p752 / PDF2, ordinary GEM definition and cross-entropy argument

Scope #likelihood-bound

For a well-defined normalized model with the support and finite-expectation conditions needed for the EM identity, the exact-current-posterior bound touches observed log likelihood at theta_t. Increasing its parameter-dependent Q term makes observed likelihood nondecreasing relative to that anchor. Stale per-case expectations or noisy estimates break that premise unless separately controlled.

supports Source: A View of the EM Algorithm that Justifies Incremental, Sparse, and Other Variants · Locator: Section2 Eq1-5, PDF3-6; contrast with Sections3 and5

Limitation #ascent-not-convergence

Nondecreasing likelihood alone does not establish a stationary parameter limit, a local maximum or a global maximum. Even strict Q increases can become summably small and approach a nonstationary point, as the saved normalized Gaussian-model example shows. Convergence requires additional update-map, regularity and progress conditions. The source theorem about a joint maximum of the free-energy objective does not turn every coordinate-ascent sequence into such a maximum.

supports Source: A View of the EM Algorithm that Justifies Incremental, Sparse, and Other Variants · Locator: Section2 Theorem2 and subsequent alternating-optimization discussion, PDF5-6; curator shrinking-step counterexample

⚠️ Curated Limitations & Negative Results

Exact posterior expectations can be expensive or unavailable. Monte Carlo, variational or numerical approximations need their own error analysis and are not certified GEM steps merely because they aim to increase a surrogate.
The original Dempster-Laird-Rubin1977 source and Wu1983 convergence proof were not newly read for this record; the fully inspected primary chapter and published correction support the scoped definition. No universal stationary-limit theorem is asserted.
No author software or historical empirical study was reproduced. Finite curator calculations illustrate stated objective and schedule boundaries; they are not a general convergence proof or an empirical speed comparison.

Artificial Intelligence Ontology (AIO) Grounding

Mapped AIO Classes

AIO Target Class	Mapping Relation	Mapping Rationale
`AIO:MachineLearningTask` Machine Learning Task	related_match	This procedure fits latent-model parameters from observed data. The inspected AIO target denotes a learning process/task; this algorithm-to-task relationship asserts neither equivalence nor an is-a relation.

Evidence Sources & Primary Literature

2 Sources

Source ID	Publication / Document	Type / Access	Reference & Checksum
`neal-source`	A View of the EM Algorithm that Justifies Incremental, Sparse, and Other Variants Radford M. Neal, Geoffrey E. Hinton	primary paper full_text	ID: https://www.cs.toronto.edu/~hinton/absps/emk.pdf sha256:6b07cbc3c578…
`correction-source`	Comments on “Efficient Training Algorithms for HMM’s Using Incremental Estimation” William Byrne, Asela Gunawardana	primary paper full_text	ID: https://www.microsoft.com/en-us/research/wp-content/uploads/2016/02/byrne00__commen_train_algor_using_increm.pdf sha256:3b92702a985c…

Curation Provenance

Curator: Codex

Method: ai_assisted

Model: GPT-6

Created: 2026-10-12T04:39:47Z

Updated: 2026-10-12T04:39:47Z

Source File: data/curated/GeneralizedExpectationMaximization.yaml