Autoregressive conditional restricted Boltzmann machine
Architectures AIMech:AutoregressiveConditionalRestrictedBoltzmannMachine DRAFT
Aliases: Autoregressive CRBM
A neural sequence architecture that models each current vector with a conditional RBM whose visible and hidden biases are affine functions of a fixed window of preceding observed vectors, without hidden-to-hidden temporal links.
Identity Notes & Concept Scope
This record is scoped to Taylor, Hinton and Roweis’s autoregressive CRBM construction for vector sequences, particularly its Gaussian-visible, binary-hidden formulation. The general label conditional RBM and acronym CRBM are not claimed as exclusive aliases because other conditional RBMs condition on arbitrary inputs or use different parameterizations. This is the observed-history specialization related to TRBM-HH; RTRBM and full TRBM also propagate hidden history. A stacked multilevel construction is not silently included in the single-layer record. All 442 pinned AIO labels/scoped synonyms and 361 preceding native IDs/names/aliases were compared, with no normalized identity match. Full-checkout fixed-string searches included ignored and hidden files in both worktree and main, returning 24 lines each. All 18 unique matched contents were read: temporal FRBM source references, model contrasts, a discovery lead and saved review context, rather than existing candidate records. All 44 semantic label/synonym hits and 21 exact AIO nodes were inspected, alongside corresponding generated records and four complete native neighbors. Previously read contextual files were checked for byte identity. Absence is bounded to this snapshot and inspected neighborhood.
Claims & Grounded Evidence
1 Claim
Mechanism
#method
Condition on preceding visible frames and compute dynamic biases d_t=c+sum_k A_k^T v_(t-k) and e_t=b+sum_k D_k^T v_(t-k), allowing different lag windows for the two sums. In the source’s unit-variance Gaussian formulation, E(v_t,h_t|history)=one-half times ||v_t-d_t|| squared minus e_t^T h_t minus v_t^T W h_t. Thus hidden probabilities are sigmoid(e_t+W^T v_t), and visible coordinates given h_t are independent unit-variance Gaussians with mean d_t+W h_t. The partition is conditional on the fixed history. For fully observed frames the hidden posterior factorizes, and likelihood gradients contrast data and conditional-model statistics; the paper approximates them by CD while holding earlier frames fixed. Its implementation description samples positive-phase binary hidden states, uses hidden probabilities for negative-phase statistics, and uses mean visible reconstructions in place of Gaussian draws. Generation starts from supplied initial frames and repeatedly samples the current conditional RBM; the original conditional training does not model the first lag-window frames.
⚠️ Curated Limitations & Negative Results
- The single-layer visible conditional depends on the fixed observed lag window. Nonlinear hidden features do not introduce an unbounded latent temporal memory path; deeper stacked models are separate compositions.
- Sections 2–2.1 explicitly give exact factorial hidden inference for fully observed frames. Section 2.2’s broad statement about omitted smoothing must not override that graph-specific result: future factors contain no earlier hidden state once the visible sequence is fixed. Missing visible data and multilevel models can require smoothing; the reported online missing-data procedure is approximate.
- Replacing Gaussian visible samples by conditional means removes conditional sampling variance and is not exact Gibbs sampling from the stated Gaussian model. Finite CD is not an exact likelihood gradient or a convergence guarantee.
- The general energy in Equation (1) includes visible standard deviations and scaled interactions; the displayed mechanism deliberately adopts the source’s sigma_i=1 convention. Rescaling arbitrary variances without transforming weights changes the distribution.
- A conditional RBM is richer than a linear autoregression, although removing the hidden interactions leaves an autoregressive Gaussian special case. Initial-frame distributions, missing-data posterior integration, phase disambiguation and transition coverage need separate modeling assumptions. No benchmark ranking or author-code certification is asserted.
Artificial Intelligence Ontology (AIO) Grounding
Mapped AIO Classes
| AIO Target Class | Mapping Relation | Mapping Rationale |
|---|---|---|
AIO:ArtificialNeuralNetwork
Artificial Neural Network
|
broad_match | The pinned AIO class is broader than this particular neural sequence architecture with nonlinear stochastic units and weighted connections. This mapping does not claim equivalence to a symmetric Boltzmann machine, a pure directed Bayesian network or an autoregressive language model. The latter is language-specific, while this model supports general vector sequences. |
AIO:RestrictedBoltzmannMachine
Restricted Boltzmann Machine
|
related_match | Each time step uses a conditional RBM with undirected visible–hidden interactions. The entire sequence construction additionally has directed history connections and a separate conditional normalizer per step; a related mapping records the component relationship without equating the whole temporal architecture to one unconditioned symmetric RBM. |
Evidence Sources & Primary Literature
2 Sources| Source ID | Publication / Document | Type / Access | Reference & Checksum |
|---|---|---|---|
method-source
|
Graham W. Taylor, Geoffrey E. Hinton, Sam Roweis
|
primary paper
full_text
|
ID: neurips-2006-1091660f3dff84fd648efe31391c5524
sha256:7613916f0695…
|
Curation Provenance
Curator: Codex
Method: ai_assisted
Model: GPT-6
Created: 2026-10-11T18:40:25Z
Updated: 2026-10-11T18:40:25Z