Nguyen–Wainwright–Jordan variational bound
Mathematical Methods AIMech:NguyenWainwrightJordanBound DRAFT
Aliases: NWJ bound, MINE-f objective, f-GAN KL bound, NWJ mutual information bound
A variational lower bound on Kullback–Leibler divergence that subtracts a product-distribution exponential critic moment from a joint-distribution critic expectation when specialized to mutual information.
Identity Notes & Concept Scope
NWJ denotes the variational objective, also called the f-GAN KL bound or MINE-f objective in the inspected follow-up. The neural MINE-f estimator is an optimization configuration using this bound, not a separate formula here. The original paper also develops density-ratio estimation algorithms; this record does not equate the mathematical bound with those kernel estimators or with MINE’s Donsker–Varadhan objective. No normalized identity match was found across all 442 pinned AIO labels/scoped synonyms and 208 preceding native IDs/names/aliases. Word-delimited fixed-string searches of both full checkouts, including ignored and hidden files, returned 57 lines each: existing MINE-f component discussion, shared primary-source references, and discovery/review context. Four native neighbors and seven exact AIO nodes with scoped synonyms and outgoing relations were inspected.
Claims & Grounded Evidence
1 Claim
Mechanism
#method
For P absolutely continuous with respect to Q and a real critic T with well-defined expectations, set L_NWJ=E_P[T]−E_Q[exp(T−1)]<=KL(P||Q). The original representation is sup_(g>0){E_P[log g]−E_Q[g]+1}; substituting g=exp(T−1) gives the displayed objective. With P=p(x,y) and Q=p(x)p(y), it lower-bounds I(X;Y). Under positive density ratios and sufficient critic capacity, T*=1+log(dP/dQ) attains equality; boundary cases may require limiting critics. For a fixed critic with integrable moments, correctly sampled empirical means estimate this expected bound without a log-of-sample-mean bias. Optimizing a neural critic changes the statistical estimation and optimization problem and does not inherit convexity merely from the original function-space formulation.
⚠️ Curated Limitations & Negative Results
- The exponential product term can have large or infinite variance. Unbiased estimation of a fixed critic’s objective is not unbiased estimation of true mutual information unless the variational gap vanishes.
- A realized finite-sample value need not stay below the true divergence. Critic fitting and evaluation on the same data introduce selection effects; finite optimization and restricted function classes add approximation error.
- The optimal critic includes a fixed normalization offset. Arbitrarily shifting a nonoptimal critic changes this objective, unlike the shift invariance of the population Donsker–Varadhan expression.
- The original constrained and penalized density-ratio programs have their own assumptions. Their consistency or convexity results are not claimed for every neural parameterization. No estimator benchmark was reproduced.
Artificial Intelligence Ontology (AIO) Grounding
Explicit AIO Snapshot Assessment: No Match
Rationale: The inspected candidates describe networks, feature-extraction tasks or self-supervised tasks, not the mathematical information bound. Shared variational terminology, an optional neural parameterization or downstream use does not establish a suitable mapping.
Search Scope: All 442 pinned AIO CLASS labels and scoped synonyms and 208 preceding native IDs/names/aliases were compared. Full fixed-string word-delimited searches of both checkouts included hidden and ignored files. Exact definitions, synonyms and outgoing relations for FeatureExtraction, VariationalAutoEncoder, AutoEncoderNetwork, RepresentationLearning, DeepNeuralNetwork, BayesianNetwork and SelfsupervisedLearning were inspected. This assessment is bounded to AIO commit 397851c547d1c4d8f06ef184f63c1d40bc86937a and this semantic neighborhood.
| Examined AIO Candidate | Rejection Reason |
|---|---|
AIO:VariationalAutoEncoder
|
A probabilistic latent autoencoder architecture is not a variational information inequality; neither implies the other. |
AIO:AutoEncoderNetwork
|
Reconstruction networks can use information objectives but do not identify this mathematical bound. |
AIO:RepresentationLearning
|
The target is a deep-network representation-learning class; it is a possible use of a bound, not the bound itself. |
AIO:DeepNeuralNetwork
|
A neural critic or decoder is one possible parameterization, not the information formula. |
AIO:FeatureExtraction
|
This preprocessing task does not specify an information estimator or bound. |
AIO:BayesianNetwork
|
Conditional distributions do not make this bound a directed graphical-model architecture. |
AIO:SelfsupervisedLearning
|
A self-generated prediction task does not specify this population information inequality. |
Evidence Sources & Primary Literature
2 Sources| Source ID | Publication / Document | Type / Access | Reference & Checksum |
|---|---|---|---|
method-source
|
XuanLong Nguyen, Martin J. Wainwright, Michael I. Jordan
|
primary paper
full_text
|
ID: https://doi.org/10.1109/TIT.2010.2068870
sha256:7f8582371c96…
|
additional-1
|
Ben Poole, Sherjil Ozair, Aäron van den Oord, Alexander A. Alemi, George Tucker
|
primary paper
full_text
|
ID: https://proceedings.mlr.press/v97/poole19a.html
sha256:38da74b0bae6…
|
Curation Provenance
Curator: Codex
Method: ai_assisted
Model: GPT-6
Created: 2026-10-11T09:02:43Z
Updated: 2026-10-11T09:02:43Z
Source File:
data/curated/NguyenWainwrightJordanBound.yaml