Black box variational inference
Algorithms AIMech:BlackBoxVariationalInference DRAFT
Aliases: BBVI
A variational-inference procedure that estimates evidence-lower-bound gradients from samples of a tractable variational distribution using its score function and model log joint, with mean-field local signals and score control variates to manage estimator variance.
Identity Notes & Concept Scope
This record covers Ranganath, Gerrish and Blei’s original BBVI Algorithms 1–2 and their score-function estimator, mean-field local signals and control variates. It is not every black-box inference method, a pathwise reparameterization estimator, a model architecture or an amortized recognition network such as NVIL. The hierarchical data-subsampling extension is outside the validated mechanism scope here. No normalized identity match was found across all 442 pinned AIO labels/scoped synonyms and 337 preceding native IDs/names/aliases. Full-checkout fixed-string searches included ignored and hidden files in worktree and main, returning 22 and 22 lines respectively. All 15 unique matched line contents were inspected in full: the preceding gradient-estimator records, their cited Bengio source, saved source-reading scopes, reviews and open discovery leads, not existing candidate records. Exact checkout counts and differences are retained in the ledger. Thirteen exact AIO CLASS nodes with scoped synonyms/comments and outgoing edges and six full native neighboring records were inspected. Absence is bounded to this snapshot and semantic neighborhood.
Claims & Grounded Evidence
1 Claim
Mechanism
#method
For fixed observed x and joint model p(x,z), choose a variational family q(z;lambda), sample z from q, and maximize L(lambda)=E_q[log p(x,z)-log q(z;lambda)] by stochastic ascent. Under the normalized-score and differentiation-under-expectation conditions, the raw gradient is E_q[grad_lambda log q(z;lambda)*(log p(x,z)-log q(z;lambda))], estimated by a Monte Carlo average. The mean-field version factors q into q_i(z_i;lambda_i) and uses for each factor the model log terms that depend on z_i together with -log q_i; unrelated terms cancel in expectation under the stated factorization. For parameter coordinate d in factor i, let h_d be the score and f_d be that score times the local log signal. A fixed control-variate coefficient gives f_d-a_d*h_d because E[h_d]=0; the population variance-minimizing coefficient is Cov(f_d,h_d)/Var(h_d) when these quantities exist and Var(h_d)>0. The source estimates coefficients from samples and averages corrected contributions before a stochastic-ascent update. Its fixed-coefficient expectation identity does not by itself justify arbitrary reuse of samples to estimate both the coefficient and corrected gradient.
⚠️ Curated Limitations & Negative Results
- Black box still requires sampling from and evaluating the variational distribution, score derivatives with respect to its parameters, and evaluation of the model log joint at sampled latent states. The method does not require differentiation through a discrete latent sample, but support and integrability conditions remain necessary.
- Raw unbiasedness requires the regularity assumptions behind differentiation under the expectation and zero expected score. A model normalization constant independent of lambda cancels from this gradient; a parameter-dependent term cannot be discarded on that basis.
- A coefficient fixed independently of the gradient samples preserves expectation when the relevant moments exist. A coefficient estimated from those same samples can correlate with the sample score and introduce bias. The paper reports separate sample counts for coefficient estimation and gradient calculation, but the inspected description does not establish an exact sampling-independence convention; no implementation behavior is asserted.
- Population-optimal control-variate scaling and conditional-expectation variance reduction do not guarantee that estimated coefficients or every additional removal of zero-mean terms reduces finite-sample variance. Zero or poorly estimated score variance requires handling beyond division by the displayed ratio.
- The practical step schedule and change-in-parameters stopping rule do not establish global optimality or exact posterior recovery. Mean-field factorization can limit the approximation. The printed hierarchical subsampling formulas need a separate weighting audit and are not curated as a verified extension here.
- No benchmark result, author-code run, general convergence proof or posterior-quality certification is included.
Artificial Intelligence Ontology (AIO) Grounding
Mapped AIO Classes
| AIO Target Class | Mapping Relation | Mapping Rationale |
|---|---|---|
AIO:ProbabilisticGraphicalModel
Probabilistic Graphical Model
|
related_match | The original method’s factor-local signals explicitly use a probabilistic model’s conditional-dependence structure and Markov blankets. The pinned target denotes the graphical-model task, while BBVI is an inference/optimization procedure; this is a method-to-modeling-context related match, not equivalence or an is-a claim. The estimator is not restricted to a Bayesian-network architecture. |
Evidence Sources & Primary Literature
2 Sources| Source ID | Publication / Document | Type / Access | Reference & Checksum |
|---|---|---|---|
method-source
|
Rajesh Ranganath, Sean Gerrish, David M. Blei
|
primary paper
full_text
|
ID: pmlr-v33-ranganath14
sha256:8f481e202eb6…
|
bbvi-supplement
|
Rajesh Ranganath, Sean Gerrish, David M. Blei
|
primary paper
full_text
|
ID: pmlr-v33-ranganath14-supp
sha256:f61d07ae80b6…
|
Curation Provenance
Curator: Codex
Method: ai_assisted
Model: GPT-6
Created: 2026-10-11T17:00:15Z
Updated: 2026-10-11T17:00:15Z
Source File:
data/curated/BlackBoxVariationalInference.yaml