AIMech AI

Stochastic Approximation Expectation Maximization

Algorithms AIMech:StochasticApproximationExpectationMaximization DRAFT
Aliases: SAEM, Stochastic Approximation EM
An incomplete-data fitting method that recursively averages simulated complete-data log objectives or sufficient statistics using a step-size sequence, then maximizes the accumulated objective to update the model parameters.
Identity Notes & Concept Scope The Delyon-Lavielle-Moulines SAEM construction fits a fixed observed data set while stochastic approximation controls integration noise. It is distinct from fresh-batch MCEM, stochastic-gradient parameter steps, parameter-iterate averaging, and online learning from newly arriving observations. A persistent MCMC simulation step is a separately scoped extension because exact conditional draws and Markovian perturbations require different arguments. All 442 pinned AIO labels/scoped synonyms and 470 preceding native IDs, names and aliases were compared with no normalized identity match. Ignored-inclusive whole-checkout fixed-string searches found 218 matched lines in each checkout; context windows for all 41 distinct matched texts were read. BayesianDataAugmentation, HiddenMarkovTree and IterativeConditionalEstimation mention applications or adjacent estimators, not these identities. All 225 semantic label/synonym rows were recomputed and verified identical to the preceding fully read inventory. Sixteen annotated AIO nodes and outgoing edges comprise fourteen verified prior readings and two new readings. Thirty-four contextual YAMLs comprise twenty-seven byte-verified prior readings and seven full new readings. Absence is bounded to this pin, maintained corpus and inspected neighborhood. No normalized name/alias matches among 442 pinned and 470 maintained records. BayesianDataAugmentation, HiddenMarkovTree and IterativeConditionalEstimation matches are applications or adjacent estimators, not duplicate generic MCEM, SAEM or MCMC-SAEM records. The inspected sufficient-statistic maximization and fixed-data scope distinguish SAEM from averaged parameter gradients, ICE and sequential-task continual learning. Persistent Markovian simulation distinguishes the MCMC extension from the exact-conditional-draw base theorem.

Claims & Grounded Evidence

4 Claims
Mechanism #recursive-expectation
Simulate latent completions under the current conditional law and update Q_k(theta)=(1-gamma_k)Q_(k-1)(theta)+gamma_k times the current sample mean of complete log objectives. Maximize Q_k to obtain the next parameter. For the stated curved exponential family, store sufficient statistics instead: s_k=s_(k-1)+gamma_k(S(z_k)-s_(k-1)), followed by theta_k=theta_hat(s_k). The one-draw construction can therefore retain past information without storing every prior completion.
Scope #step-sizes-and-stationary-set
The single-draw stationary-set theorem requires a smooth curved-exponential complete model and maximizer map, exact conditional simulation given the past, finite continuous second moments, step sizes in [0,1] with divergent sum and finite sum of squares, and almost-sure containment of the statistic iterates in a compact subset of their domain. Under those hypotheses the distance to the stated stationary sets tends to zero. Step-size conditions alone do not supply stability, point convergence, a global maximum or observed-likelihood ascent at every iteration.
Scope #averaging-and-estimation-boundaries
Recursive averaging of the expected objective precedes its maximization. Optional averaging of fitted parameter iterates is a separate operation with additional local stability, moment and step-size conditions. Averaging statistics before a nonlinear maximizer need not equal averaging parameter values. Simulation-iteration uncertainty around a fixed-data solution is distinct from sampling uncertainty of the statistical estimator.
Limitation #source-sign-and-zero-step
In Eq6, gamma=0 freezes the accumulated objective, while gamma=1 replaces it by the current Monte Carlo objective; the paragraph identifying gamma=0 with ordinary stochastic EM is inconsistent with that recursion. The 1999 covariance prose also reverses the Hessian sign: its H=G-Delta Delta^T estimates the observed log Hessian, while observed information is -H, as explicitly used in the 2004 follow-up. These printed statements are not used as implementation rules.

⚠️ Curated Limitations & Negative Results

Artificial Intelligence Ontology (AIO) Grounding

Mapped AIO Classes
AIO Target Class Mapping Relation Mapping Rationale
AIO:MachineLearningTask
Machine Learning Task
related_match This algorithm fits statistical-model parameters from observed data. The inspected AIO target denotes a learning task/process, so this is related algorithm-to-task context, not equivalence or a subclass assertion. The pinned TrainingStrategy and transformation-based DataAugmentation nodes do not identify statistical latent-data integration, and fixed-data iteration is not the pinned sequential-task ContinualLearning network.

Evidence Sources & Primary Literature

2 Sources
Source ID Publication / Document Type / Access Reference & Checksum
saem-source
Bernard Delyon, Marc Lavielle, Eric Moulines
primary paper
full_text
ID: 10.1214/aos/1018031103
sha256:f2c7a8005a1d…
mcmc-source
Estelle Kuhn, Marc Lavielle
primary paper
full_text
ID: 10.1051/ps:2004007
sha256:48f3b9861570…

Curation Provenance

Curator: Codex
Method: ai_assisted
Model: GPT-6
Created: 2026-10-12T04:03:36Z
Updated: 2026-10-12T04:03:36Z
Source File: data/curated/StochasticApproximationExpectationMaximization.yaml