Real-valued neural autoregressive density estimator
Architectures AIMech:RealValuedNeuralAutoregressiveDensityEstimator DRAFT
Aliases: RNADE, Real-valued autoregressive density estimator
A fully visible neural autoregressive density architecture for real-valued vectors that shares input-to-hidden parameters across ordered scalar conditional mixture-density models.
Identity Notes & Concept Scope
This record denotes Uria, Murray and Larochelle’s original RNADE family. It extends binary NADE by replacing Bernoulli outputs with real-valued mixture conditionals. Deterministic prefix features and NADE-style input sharing distinguish it from separate untied conditional networks and from a stochastic latent RBM. Mixture-of-Gaussians and mixture-of-Laplace outputs are reported variants, not identity aliases for every Gaussian or Laplace mixture. No normalized identity match was found across all 442 pinned AIO labels/scoped synonyms and 349 preceding native IDs/names/aliases. Full-checkout fixed-string searches included ignored and hidden files in worktree and main, returning 5 and 5 lines respectively. All 3 unique matched line contents were inspected in full: the prior NADE-variant lead, RNADE source URL and previous review exclusion, not existing candidate records. Exact checkout counts and differences are retained in the ledger. Seventeen exact AIO CLASS nodes with scoped synonyms/comments and outgoing edges and five full native neighboring records were inspected. Absence is bounded to this snapshot and semantic neighborhood.
Claims & Grounded Evidence
1 Claim
Mechanism
#method
For an ordered real-valued vector x, evaluate p(x)=product_d p(x_d|x_<d). Start a_1=c and update a_(d+1)=a_d+x_d W[:,d] when another conditional remains. At each position, a learned scalar rho_d rescales the shared prefix preactivation; sigmoid or rectified-linear units produce deterministic hidden features h_d. Separate position-specific output matrices map those features to scalar-mixture parameters. In RNADE-MoG, softmax gives component weights, linear outputs give means, and exponentials give positive standard deviations. Summing the normalized scalar Gaussian components gives each conditional density; multiplying the conditionals yields a normalized joint without a global partition-function estimate. Complete-vector log densities and their derivatives are directly evaluable; prefix recurrence and fixed hidden/component sizes give cost linear in the number of dimensions, with output work O(DHK). Ancestral generation samples each scalar from the mixture conditioned on previously generated values. The experimental mean-output gradient rescaling by component standard deviation is a training heuristic, separate from the exact log-density derivative. The paper also substitutes Laplace mixture conditionals.
⚠️ Curated Limitations & Negative Results
- The variable ordering, conditional family, hidden capacity, component count and optimization restrict the fitted density. Exact density evaluation does not make arbitrary missing-variable posterior inference a forward pass or establish data-model agreement.
- Linear scaling in dimensionality holds with fixed hidden and mixture sizes; it does not omit their cost. The final prefix update is unnecessary, consistent with the declared H by (D-1) input-weight matrix.
- The supplement labels several log-density derivative formulas as derivatives of p rather than log p. Its Algorithm 2 also uses a log-shaped expression with a positive squared-residual term as if it were a component density in responsibilities, and has prefix/index ambiguities. A literal finite example produces invalid responsibilities. The curated mechanism follows the main-paper normalized densities and prefix equations; this is not a claim about author-code behavior.
- The experimental standard-deviation gradient rescaling is not the raw likelihood gradient or Newton’s variance scaling. Exponential scales remain susceptible to approaching zero; regularization and model selection matter.
- No performance ranking, convergence guarantee, exact posterior claim or full executable implementation validation is included.
Artificial Intelligence Ontology (AIO) Grounding
Mapped AIO Classes
| AIO Target Class | Mapping Relation | Mapping Rationale |
|---|---|---|
AIO:BayesianNetwork
Bayesian Network
|
broad_match | The primary paper explicitly gives the ordered real-valued factorization as a fully visible Bayesian network. The AIO target is broader than this shared-parameter neural mixture-conditional architecture. Hidden neural activations parameterize factors and are not unobserved random variables. A time-series autoregression or language-model label does not define the same identity. |
Evidence Sources & Primary Literature
2 Sources| Source ID | Publication / Document | Type / Access | Reference & Checksum |
|---|---|---|---|
method-source
|
Benigno Uria, Iain Murray, Hugo Larochelle
|
primary paper
full_text
|
ID: https://proceedings.neurips.cc/paper/2013/hash/53adaf494dc89ef7196d73636eb2451b-Abstract.html
sha256:37ae0b99e5c3…
|
supplement-source
|
Benigno Uria, Iain Murray, Hugo Larochelle
|
primary paper
full_text
|
ID: https://proceedings.neurips.cc/paper/2013/hash/53adaf494dc89ef7196d73636eb2451b-Abstract.html#supplement
sha256:011b51237254…
|
Curation Provenance
Curator: Codex
Method: ai_assisted
Model: GPT-6
Created: 2026-10-11T17:49:48Z
Updated: 2026-10-11T17:49:48Z