Mixed memory Markov model
Model Families AIMech:MixedMemoryMarkovModel DRAFT
A family of compact temporal probability models whose component transition distributions are convex mixtures of transition tables indexed by a single selected predecessor lag or component.
Identity Notes & Concept Scope
This record covers the mixed-memory construction of Saul and Jordan, including higher-order scalar, multicomponent first-order and hidden-state extensions. Mixed memory means a latent selector chooses one predecessor for a component distribution. It does not mean external neural memory, a mixture selecting an entire HMM, independent factorial chains, or an unrestricted full higher-order transition table. EM and coordinate Viterbi are estimation/inference procedures, not aliases for the model. No normalized identity match was found across all 442 pinned AIO labels/scoped synonyms and 379 preceding native IDs/names/aliases for the four researched candidate names. Both full-checkout fixed-string searches included ignored and hidden files, with 7 and 7 matching lines. All 6 unique matched contents were read: pending leads and preceding source/review context, not existing records of these three new identities. Expanded source IDs, plurals, Gibbs-machine and mixture-transition terminology were included. All 63 semantic label/synonym hits, 17 exact fully annotated AIO CLASS nodes with outgoing edges, their generated YAMLs and six native neighbors were read or byte-verified after prior complete reading. Markov mixtures of experts substantially overlaps the existing IOHMM family, so no fourth ID or unconditional alias is created. Absence is bounded to the pinned vocabulary and inspected neighborhood.
Claims & Grounded Evidence
1 Claim
Mechanism
#method
For a scalar k-order sequence, write p(i_t|i_(t-1),...,i_(t-k))=sum_mu psi(mu) a_mu(i_t|i_(t-mu)), with nonnegative selector weights summing to one and each transition table normalized over the current state. Given observed predecessor and current values, a selector responsibility is proportional to psi(mu) a_mu(i_t|i_(t-mu)); parameter re-estimation can use these expected selector-weighted counts. For a vector state I_t with components nu, the first-order multicomponent form is p(I_t|I_(t-1))=product_nu sum_mu psi_nu(mu) a_nu,mu(i_t^nu|i_(t-1)^mu). Thus current components are conditionally independent given the complete previous vector, despite possible marginal dependence. The hidden extension uses this transition and an emission p(J_t|I_t)=product_nu sum_mu phi_nu(mu) b_nu,mu(j_t^nu|i_t^mu). Fully observed parameter counts scale quadratically rather than exponentially in the number of components for fixed alphabet size; exact hidden posterior inference can still be exponential. The paper instead illustrates a coordinate Viterbi approximation that fixes other hidden chains while optimizing one chain path.
⚠️ Curated Limitations & Negative Results
- A convex sum of single-predecessor effects cannot represent every higher-order interaction; for example an unrestricted XOR dependence on two binary predecessors is not additive in this way. Compact parameter counts do not establish universal expressiveness.
- The displayed vector transition explicitly assumes conditional independence of current components given the previous vector. The source’s broad wording about avoiding strong conditional-independence assumptions must not override its product factorization.
- Selector weights are properties of a fitted parameterization, not uniquely identified causal influence or a generally valid measure of correlation. Initial histories and observed-window boundaries must be specified when counting transitions.
- In the hidden extension, exact EM needs joint posterior expectations of selectors and hidden-state indicator products. Equations (20) and (22) display hidden indicators outside the expectation, which is not a general full-posterior EM formula; clamping a path changes the calculation. Equation (17) also reverses emission arguments relative to Equation (16).
- Coordinate Viterbi improves or preserves the joint path score under an exact single-chain update, but need not reach global MAP or optimize the marginal observed likelihood. Ties and stopping rules require care.
- The Bach example illustrates the authors’ approximate procedure. Figure 2 uses joint path log probability, not exact marginal likelihood. No accuracy, causal discovery, training reproduction or general scaling guarantee for hidden inference is curated.
Artificial Intelligence Ontology (AIO) Grounding
Mapped AIO Classes
| AIO Target Class | Mapping Relation | Mapping Rationale |
|---|---|---|
AIO:BayesianNetwork
Bayesian Network
|
broad_match | For a finite time window and specified initial history, normalized conditional factors and latent predecessor selectors form a directed acyclic probabilistic model. This compact mixed-memory parameterization is narrower than the pinned BayesianNetwork class. |
AIO:MarkovChain
Markov Chain
|
related_match | The multicomponent case has a first-order joint vector state, while the scalar case can use multiple predecessor lags and the hidden extension marginalizes states. The pinned first-order MarkovChain definition is related temporal structure, not an exact identity or an unqualified parent for every observed process. |
Evidence Sources & Primary Literature
1 Source| Source ID | Publication / Document | Type / Access | Reference & Checksum |
|---|---|---|---|
method-source
|
Lawrence K. Saul, Michael I. Jordan
|
primary paper
full_text
|
ID: https://proceedings.mlr.press/r1/saul97a.html
sha256:a38f538362ef…
|
Curation Provenance
Curator: Codex
Method: ai_assisted
Model: GPT-6
Created: 2026-10-11T20:01:01Z
Updated: 2026-10-11T20:01:50Z
Source File:
data/curated/MixedMemoryMarkovModel.yaml