Supplemented Expectation Conditional Maximization
Algorithms AIMech:SupplementedExpectationConditionalMaximization DRAFT
Aliases: Supplemented ECM, SECM
A numerical supplement to expectation conditional maximization that recovers observed-data asymptotic parameter covariance from expected complete-data information and the local derivatives of both the ECM update and its frozen-expectation conditional-maximization sweep.
Identity Notes & Concept Scope
SECM extends supplemented EM to an ECM fit. Its additional CM derivative accounts for incomplete joint maximization within each ECM iteration. It is a covariance-estimation procedure, distinct from ECM fitting, AECM, stochastic EM and an ordinary EM supplement applied without this correction. The supplemented-CM helper is described within this procedure rather than introduced as an ambiguous SCM record. All 442 pinned AIO labels/scoped synonyms and 466 preceding native IDs, names and aliases were compared. The sole normalized search-term collision, bare SAGE, identifies the existing sparse additive text model; the new optimization method uses qualified aliases. Ignored-inclusive fixed-string searches of both full checkouts returned 436 and 459 line instances. Context windows for all 143 distinct normalized matched texts were inspected; duplicate ledger/review mentions, unrelated environment strings and the stochastic-EM lead do not establish these four method identities. All 225 semantic label/synonym rows and 17 annotated AIO nodes with outgoing edges were inspected; sixteen nodes were byte-verified unchanged after prior reading and UncertaintyBias was newly read. Thirty-one contextual YAMLs comprise 17 AIO and 14 native records, with 27 verified prior readings and four full new readings; the existing sparse text SAGE was also reread. Absence is bounded to this pin, maintained corpus and inspected neighborhood.
Claims & Grounded Evidence
4 Claims
Mechanism
#separate-ecm-and-cm-derivatives
At the converged interior fit, estimate D_ECM by perturbing one input coordinate and applying one E-step followed by the full ordered CM sweep. Estimate D_CM using the same perturbed inputs but only the CM sweep with the converged conditional expected sufficient statistics held fixed, or compute it analytically from complete information and constraint derivatives. Recomputing an E-step inside this second calculation would estimate the wrong map. Derivatives use input-coordinate rows and output-coordinate columns.
Mechanism
#corrected-covariance-product
Let Vcom=Ic^(-1), with Ic the expected complete-data information at the fit. The identity I-D_ECM=(I-D_EM)(I-D_CM) gives Vobs=Vcom(I-D_CM)(I-D_ECM)^(-1). The missing-data increment is Vcom(D_ECM-D_CM)(I-D_ECM)^(-1). Preserve matrix order. D_CM=0 reduces to supplemented EM; substituting D_ECM directly into the EM-only formula generally does not recover the observed covariance.
Scope
#multicycle-fit-versus-probe
A multicycle-ECM fit may supply the converged parameter and nearby perturbation values, but the stated SECM calculation still differentiates the ordinary single-E ECM map and its frozen-Q CM sweep. Extra E-steps used while locating the fit cannot simply be retained inside the derivative probes without changing the matrix identity.
Limitation
#diagnostic-not-certification
The theoretically symmetric missing-data covariance increment is computed from separately estimated matrices, making appreciable asymmetry a useful indication of numerical or implementation errors. Tight fit tolerances, stable parameterization and meaningful perturbations matter. Symmetry is not sufficient proof of correct fitting, covariance or uncertainty calibration; do not force symmetry merely to conceal a failed diagnostic.
⚠️ Curated Limitations & Negative Results
- The regular interior and invertibility assumptions exclude treating a boundary or singular fit with the same inverse-information formula. A large missing-information fraction amplifies errors in map differentiation.
- The straightforward sufficient-statistic routines in Section 3 assume an exponential-family complete model. The method needs expected complete information as well as ECM code; its cost and numerical precision depend on parameter dimension and conditioning.
- The p.67 example prose refers to displayed covariance blocks where the general Eq 2.4.4 specifies information blocks. The record follows the general information-matrix formula, not that inconsistent prose.
- A fitting sequence that reaches a coordinate exactly can make ratio probes undefined. The no-missing-information subcase and reduced parameterizations require the source’s additional handling.
- All 21 primary-paper pages were read. Finite Gaussian examples verify both map derivatives, noncommuting matrix order and failure of symmetry as sufficient certification. The censored-normal and contingency-table studies, original software and interval coverage were not reproduced.
Artificial Intelligence Ontology (AIO) Grounding
Explicit AIO Snapshot Assessment: No Match
Rationale: This method computes asymptotic parameter covariance by supplementing an EM-type fit with expected complete information and update-map derivatives. The inspected pin supplies no fitting-uncertainty or observed-information method counterpart. A generic task, fitted model, preprocessing operation or uncertainty-related bias would conflate the inferential calculation with a different kind of concept.
Search Scope: All 442 pinned CLASS labels and scoped synonyms,466 preceding native identities and ignored-inclusive full-checkout searches;225 semantic label/synonym rows and 17 annotated AIO nodes with outgoing edges were inspected. Assessment is bounded to the recorded AIO snapshot.
| Examined AIO Candidate | Rejection Reason |
|---|---|
AIO:MachineLearningTask
|
The target is the learning process/task. Supplemented covariance computation is a general inferential calculation after fitting, not itself a learned-task identity; that broad association alone is not used to force a link. |
AIO:Model
|
The calculation does not define a distribution/model family or a trained model instance. |
AIO:TrainingStrategy
|
This pinned node is a preprocessing strategy, not an inverse-information or numerical covariance procedure. |
AIO:RegressionAnalysis
|
Regression is only one possible application; the procedure applies to other incomplete-data likelihood models. |
AIO:DataAugmentation
|
The pinned term enriches training examples by transformations. Conditional latent-data information in EM is a different construction. |
AIO:UncertaintyBias
|
This bias favors groups better represented in training because of lower prediction uncertainty. It does not denote parameter covariance estimation. |
AIO:EvaluationBias
|
This is a bias in evaluation; a numerical covariance calculation is not such a bias or a predictive benchmark. |
Evidence Sources & Primary Literature
1 Source| Source ID | Publication / Document | Type / Access | Reference & Checksum |
|---|---|---|---|
method-source
|
David A. van Dyk, Xiao-Li Meng, Donald B. Rubin
|
primary paper
full_text
|
ID: https://www.ma.imperial.ac.uk/~dvandyk/Research/95-sinica-secm.pdf
sha256:31ec9fba0523…
|
Curation Provenance
Curator: Codex
Method: ai_assisted
Model: GPT-6
Created: 2026-10-12T03:40:12Z
Updated: 2026-10-12T03:45:56Z