Temporal restricted Boltzmann machine
Architectures AIMech:TemporalRestrictedBoltzmannMachine DRAFT
Aliases: TRBM
A neural sequence architecture that multiplies time-indexed conditional RBM distributions whose visible and hidden biases depend on preceding visible and hidden states through shared temporal weights.
Identity Notes & Concept Scope
This record covers the original single-layer temporal RBM construction of Sutskever and Hinton, with its finite lag window and omitted-edge special cases. The full architecture has visible-to-visible, visible-to-hidden and hidden-to-hidden history connections. It is distinct from temporal FRBM’s factored categorical-word construction, RTRBM’s deterministic recurrent state, and the observed-history-only autoregressive CRBM. Multilayer stacking is an extension rather than part of this single-layer identity; a filtering algorithm does not define the latent probabilistic model. All 442 pinned AIO labels/scoped synonyms and 361 preceding native IDs/names/aliases were compared, with no normalized identity match. Full-checkout fixed-string searches included ignored and hidden files in both worktree and main, returning 24 lines each. All 18 unique matched contents were read: temporal FRBM source references, model contrasts, a discovery lead and saved review context, rather than existing candidate records. All 44 semantic label/synonym hits and 21 exact AIO nodes were inspected, alongside corresponding generated records and four complete native neighbors. Previously read contextual files were checked for byte identity. Absence is bounded to this snapshot and inspected neighborhood.
Claims & Grounded Evidence
1 Claim
Mechanism
#method
For binary vectors v_t and h_t, set the visible bias a_t=a+sum_k A_k^T v_(t-k) and hidden bias b_t=b+sum_k B_k^T h_(t-k)+sum_k C_k^T v_(t-k). Define p(v_t,h_t|history)=exp(v_t^T C_0 h_t+a_t^T v_t+b_t^T h_t)/Z_t(history), and multiply these normalized factors over time using shared parameters and the source’s learned initial contributions for unavailable earlier states. Conditional on observed current and previous visibles and known previous hidden states, the current hidden posterior factorizes with probabilities sigmoid(C_0^T v_t+b_t). Unknown previous hidden states generally require marginalization; the proposed online filter instead substitutes their inferred probability vectors in b_t. Learning treats these approximate filtered states as context and uses per-step contrastive-divergence updates. The original paper distinguishes this approximation from smoothing with future observations and from an exact likelihood gradient.
⚠️ Curated Limitations & Negative Results
- The conditional partition depends on history, including previous hidden states. It is not one sequence-independent global constant, and exact posterior Gibbs updates can require ratios of such partitions.
- A factorial posterior given known latent history does not imply factorial or exact filtering after uncertain history is marginalized. Mean substitution through nonlinear normalized factors is generally approximate, and future observations can revise earlier hidden-state posteriors.
- The paper explicitly notes that changing its parameter-dependent approximate posterior can reduce the variational bound. Finite CD, mean visible reconstructions and ignoring future evidence provide no monotonic bound or likelihood guarantee.
- TRBM-HH denotes the variant with hidden-to-hidden links omitted, not one retaining only those links. For fully observed visible sequences its hidden posterior is factorial; missing observations and multilayer compositions change that inference problem.
- The displayed construction has no past-hidden-to-current-visible links; a footnote mentions those as a possible extension. Gaussian visible units, deeper compositions and different lag structures require their own stated assumptions. No empirical ranking, long-memory guarantee or author-code behavior is certified.
Artificial Intelligence Ontology (AIO) Grounding
Mapped AIO Classes
| AIO Target Class | Mapping Relation | Mapping Rationale |
|---|---|---|
AIO:ArtificialNeuralNetwork
Artificial Neural Network
|
broad_match | The pinned AIO class is broader than this particular neural sequence architecture with nonlinear stochastic units and weighted connections. This mapping does not claim equivalence to a symmetric Boltzmann machine, a pure directed Bayesian network or an autoregressive language model. The latter is language-specific, while this model supports general vector sequences. |
AIO:RestrictedBoltzmannMachine
Restricted Boltzmann Machine
|
related_match | Each time step uses a conditional RBM with undirected visible–hidden interactions. The entire sequence construction additionally has directed history connections and a separate conditional normalizer per step; a related mapping records the component relationship without equating the whole temporal architecture to one unconditioned symmetric RBM. |
Evidence Sources & Primary Literature
2 Sources| Source ID | Publication / Document | Type / Access | Reference & Checksum |
|---|---|---|---|
method-source
|
Ilya Sutskever, Geoffrey Hinton
|
primary paper
full_text
|
ID: pmlr-v2-sutskever07a
sha256:8b8b19704c75…
|
posterior-context
|
Ilya Sutskever, Geoffrey Hinton, Graham Taylor
|
primary paper
full_text
|
ID: neurips-2008-9ad6aaed513b73148b7d49f70afcfb32
sha256:7ef2f7679831…
|
Curation Provenance
Curator: Codex
Method: ai_assisted
Model: GPT-6
Created: 2026-10-11T18:40:25Z
Updated: 2026-10-11T18:40:25Z
Source File:
data/curated/TemporalRestrictedBoltzmannMachine.yaml