Helmholtz machine
Architectures AIMech:HelmholtzMachine DRAFT
A hierarchical probabilistic neural model that pairs a top-down generative model with a separately parameterized bottom-up recognition model approximating the distribution of hidden causes given observed input.
Identity Notes & Concept Scope
This record covers the original paired generative/recognition model family in Dayan and colleagues’ paper. Its original layered binary construction admits both a deterministic mean-field-inspired training approximation and a stochastic wake-sleep procedure; the architecture is not synonymous with either training rule. The generative and recognition parameters are separate, rather than tied symmetric connections of a Boltzmann machine. Related later variational autoencoders and deep belief networks are not treated as identity aliases. No normalized identity match was found across all 442 pinned AIO labels/scoped synonyms and 340 preceding native IDs/names/aliases. Full-checkout fixed-string searches included ignored and hidden files in worktree and main, returning 24 and 24 lines respectively. All 17 unique matched line contents were inspected in full: NVIL method distinctions, source-reading scopes, prior reviews and the carried wake-sleep discovery lead, not existing candidate records. Exact checkout counts and differences are retained in the ledger. Nine exact AIO CLASS nodes with scoped synonyms/comments and outgoing edges and six full native neighboring records were inspected. Absence is bounded to this snapshot and semantic neighborhood.
Claims & Grounded Evidence
1 Claim
Mechanism
#method
A top-down directed model p_theta(x,h) assigns probabilities to observed inputs and hidden configurations; a separately parameterized bottom-up model q_phi(h|x) produces a tractable approximate posterior. In the original layered binary construction, unit states in a layer are conditionally independent given the adjacent driving layer, with independent top-layer generative priors. Recognition proceeds through an acyclic bottom-up pass. The variational free energy is E_q[log q(h|x)-log p(x,h)], and log p(x) equals its negative plus KL(q(h|x)||p(h|x)); the negative free energy is therefore a lower bound under the normalized-model and support conditions. The paper describes stochastic recognition samples and, separately, a deterministic approximation that substitutes recognition means for sampled binary activities and neglects associated correlations. Its stochastic training alternative uses wake-sleep. The generative conditional need not be sigmoid: Appendix A considers noisy-or and other imaging rules. These choices do not change the defining presence of distinct generative and recognition models.
⚠️ Curated Limitations & Negative Results
- The tractable recognition family need not contain the true posterior, even when the directed generative model factors locally. Explaining-away dependencies can obstruct exact posterior matching.
- The deterministic substitution of mean activations is an additional approximation, not exact evaluation of every nonlinear expectation in the stochastic model. The exact free-energy identity does not validate all approximation formulas or practical training updates.
- Wake-sleep is a training option, and its sleep recognition step is not generally the data-ELBO gradient. A shared variational motivation does not establish a single objective decreased by all model variants.
- The inspected author manuscript’s Equation (16) prints negative free energy as a positive sum of Bernoulli KL terms, conflicting with the preceding sign convention. The curated identity uses the explicit Section 2 decomposition; Appendix B derivative implementation was not audited and that printed approximate formula is not adopted.
- The record describes a reusable architecture family, not a trained density estimator or a verified biological explanation. No exact posterior, universal compression gain or empirical ranking is asserted.
Artificial Intelligence Ontology (AIO) Grounding
Mapped AIO Classes
| AIO Target Class | Mapping Relation | Mapping Rationale |
|---|---|---|
AIO:BayesianNetwork
Bayesian Network
|
related_match | The original architecture includes a directed acyclic generative model and a distinct directed recognition model. The AIO target captures the directed probabilistic model component, whereas the native identity includes the paired-model arrangement and inference role. A related match avoids equating the entire pair with a single generative DAG or with an undirected recurrent network. |
Evidence Sources & Primary Literature
2 Sources| Source ID | Publication / Document | Type / Access | Reference & Checksum |
|---|---|---|---|
method-source
|
Peter Dayan, Geoffrey E. Hinton, Radford M. Neal, Richard S. Zemel
|
primary paper
full_text
|
ID: doi:10.1162/neco.1995.7.5.889
sha256:317a1ee43ccc…
|
wake-sleep-source
|
Geoffrey E. Hinton, Peter Dayan, Brendan J. Frey, Radford M. Neal
|
primary paper
full_text
|
ID: doi:10.1126/science.7761831
sha256:a63370a3bd28…
|
Curation Provenance
Curator: Codex
Method: ai_assisted
Model: GPT-6
Created: 2026-10-11T17:11:00Z
Updated: 2026-10-11T17:11:00Z
Source File:
data/curated/HelmholtzMachine.yaml