Stochastic Expectation Maximization
Algorithms AIMech:StochasticExpectationMaximization DRAFT
Aliases: Stochastic EM, SEM (stochastic expectation maximization)
A finite-mixture fitting algorithm that draws one hard component assignment for each observation from its current posterior responsibilities, fits the resulting completed-data model, and recomputes the responsibilities for the next iteration.
Identity Notes & Concept Scope
This record denotes the Celeux-Diebolt stochastic-completion construction. It is a single-completion specialization of Monte Carlo EM and corresponds algebraically to a one-draw, unit-weight replacement in the SAEM recursion, rather than decreasing-step recursive averaging. Its named stochastic process and stationary-summary interpretation differ from CEM posterior-mode assignment, supplemented-EM covariance estimation, general ICE estimator averaging and Bayesian sampling of model parameters. Bare SEM is not asserted as a unique alias. No normalized identity matches among 442 pinned and 473 maintained records, including the bare acronym probes. Existing supplemented EM and cross-entropy optimization are distinct methods; qualified aliases avoid claiming SEM or CEM as unique names. ICE, MCEM, SAEM, deterministic tempered EM, simulated annealing and k-means provide neighboring but different scopes. Existing stochastic-EM frontier and incidental references are not records of the new identities. The absence assessment is bounded to this snapshot and corpus. Identity-search counts: 227 matched lines in the worktree and 245 in main; all 137 distinct matching-text windows were read, with ignored and hidden files included.
Claims & Grounded Evidence
4 Claims
Mechanism
#random-completion
For current weights and component densities, normalize p_k f_k(x_i) over k to obtain responsibilities t_ik. Independently draw a categorical label for each observation under t_i, then maximize the completed-data log likelihood. With unrestricted mixture weights the update is the sampled class count divided by N; each component parameter uses its assigned observations and the selected distribution family. Recompute responsibilities from the fit. The original cyclic S-M-E presentation is equivalent to E-S-M when starting from current parameters.
Scope
#process-and-summary
Random complete-data refitting can retain fluctuations on a fixed observed data set. The source proposes summary means and variances after a stationary regime, rather than equating every iterate with a converged point estimate. Its stated asymptotic result is restricted to two components with an unknown mixing proportion and refers to an earlier technical proof; the general extension is conjectural. A reliable universal burn-in cutoff, unbiased estimator or global maximum is not established.
Scope
#small-class-policy
The stated baseline restarts when a sampled class has fewer than N*c(N,d) observations. A later practical variant instead deletes a class with count at most d and continues with one fewer component. Restart and deletion define different transition and model-order behavior. Neither automatic component-count recovery nor ergodicity follows from the unprotected assignment/count formula: a zero weight can make that component unreachable.
Limitation
#likelihood-and-partition
A sampled hard partition is not the actual posterior-responsibility matrix. The entropy identity for the actual posterior does not imply equality of completed and marginal likelihood for arbitrary sampled labels, contrary to the report prose following Eq4.4. The saved normalized Bernoulli-mixture example has a positive-probability nonempty completion whose fitted mixing weight decreases marginal likelihood. Per-step ordinary EM ascent therefore cannot be inherited by this stochastic update.
⚠️ Curated Limitations & Negative Results
- Source stationarity claims depend on the complete transition construction, admissible fits and recurrence assumptions. The original 1985 English article, 1984 proof report and distinct 1986 INRIA RR563 were not fully inspected; generic stationary normality, point convergence or global convergence is not certified.
- Model-specific completed-data estimators matter. Empty or undersized classes, unbounded Gaussian fits and identifiability up to label permutation require explicit treatment. The raw formula alone does not specify those policies.
- The 1986 numerical tables use 1000 post-stationary iterations, not 1000 independent data sets; their mean-plus/minus-SD summaries are not calibrated confidence intervals. Printed component-weight, mean-sign and component-location discrepancies remain unresolved, so no quantitative superiority or recovery claim is curated.
- The optional posterior-threshold classification can leave points unassigned; at threshold one half, tied probabilities can place a point in more than one set. It is a separate readout from the stochastic fitting transition.
- No author implementation or historical numerical experiment was reproduced. Finite curator calculations check formula boundaries, not empirical performance or a repaired convergence proof.
Artificial Intelligence Ontology (AIO) Grounding
Mapped AIO Classes
| AIO Target Class | Mapping Relation | Mapping Rationale |
|---|---|---|
AIO:MachineLearningTask
Machine Learning Task
|
related_match | The procedure estimates finite-mixture parameters from observed unlabeled data. The AIO target denotes a learning task/process; this algorithm-to-task relation is not equivalence or a subclass assertion. |
Evidence Sources & Primary Literature
2 Sources| Source ID | Publication / Document | Type / Access | Reference & Checksum |
|---|---|---|---|
sem-source
|
Gilles Celeux, Jean Diebolt
|
primary paper
full_text
|
ID: RSA_1986__34_2_35_0
sha256:ba2086b4746b…
|
cem-source
|
Gilles Celeux, Gérard Govaert
|
author technical report
full_text
|
ID: inria-00075196v1
sha256:30bd7f969d0f…
|
Curation Provenance
Curator: Codex
Method: ai_assisted
Model: GPT-6
Created: 2026-10-12T04:22:35Z
Updated: 2026-10-12T04:22:35Z
Source File:
data/curated/StochasticExpectationMaximization.yaml