Product of hidden Markov models

⚠️ Curated Limitations & Negative Results

• The partition function depends on model parameters and sequence length. Separate forward–backward computations evaluate the unnormalized product and complete-data hidden posteriors; they do not by themselves give the product’s global normalizer or exact likelihood. Small finite models can be enumerated, but this does not establish scalable tractability.
• When observations are missing or future observations are integrated out, the experts can become dependent. Exact separate posterior inference given a complete sequence is not a claim of equally simple causal prediction from a prefix.
• CD learning is approximate, and the customary short-chain update is not generally an exact likelihood or contrastive-objective gradient. The Brown paper’s initial pseudocode writes probability derivatives where its final update uses log derivatives; this record follows the log-score update supported by the later explicit formulation.
• Brown Equation (3) places a derivative outside a posterior expectation. The score identity used here means the expectation of the log-joint derivative at the current posterior, without differentiating through that posterior. The shared deterministic emission-feature extension is optional, not an extra required stochastic latent layer.
• Taylor Section 3.3 estimates normalization with an AIS path that tempers emission potentials inside hidden-path sums. This is generally different from simply raising whole HMM marginal likelihoods to fractional powers. Approximate normalizer estimates, and their logarithms, are not exact log likelihoods or a universal reliability guarantee.
• The papers’ language, rainfall and dance examples do not establish universal superiority. No author code, training experiment or AIS implementation was reproduced.