Amortized Stein variational gradient descent

⚠️ Curated Limitations & Negative Results

• A target score requires a differentiable continuous target on the relevant domain, finite positive normalization and regularity. An unknown multiplicative constant independent of the sampled variable cancels; arbitrary improper energies do not thereby become distributions.
• The infinitesimal Stein/KL argument concerns suitable population densities, moments, boundary integration and valid perturbations. A finite empirical particle measure is atomic; a finite/discrete-seed simulator can also have singular output. The algorithm can compute directions without an output density, but that alone does not make its population KL finite or justify density-based entropy identities.
• Finite sampler capacity, restricted tangent directions, rank-deficient Jacobians, nonlinear fitting error, minibatch error and finite step size separate the parameter update from ideal unconstrained SVGD. A least-squares projection may have nonunique minimizers. Even a full-column-rank but restricted parameterization can have a stationary fitting update while the unrestricted Stein field is nonzero. No universal monotone KL descent, exact posterior recovery or global convergence is adopted.
• The 2016 v1/v2 Eq.9 has eta_new=eta+epsilon*Delta_eta. The 2017 proceedings and arXiv:1707.06626v2 Eq.14 print a minus sign while minimizing ||J*delta-phi||^2 and retaining positive Eq.15. The fitting target and first-order derivation support plus for this delta definition. Retain the printed discrepancy and use positive 2016 Eq.10/2017 Eq.15 for the practical step; no published erratum or inspected author implementation is claimed.
• Unit diagonal and zero first-argument diagonal derivative give the exact one-particle score-ascent reduction, as with a normalized RBF. Other kernels can change the scale or add a derivative. The 2016 p4 single-particle sentence is visibly missing the condition value; the 2017 p3 prints derivative=0 but unit-diagonal scaling still matters. Kernel richness, boundary conditions and positive nondegenerate bandwidth matter; repulsion does not guarantee all modes or prevent coincident outputs.
• The projected Stein field is not asserted to be the exact ordinary parameter gradient of KL. The paper relates it to a kernel-smoothed score difference under Stein conditions; this does not remove smoothing, projection or family restrictions. The output target is held fixed in the regression derivative, rather than differentiating through the target as an additional objective.
• SteinGAN replaces a model expectation with imperfect sampler outputs, so finite alternating updates are not automatically exact or unbiased MLE. Its gamma-weighted stabilization changes the objective; proper prior normalization is not established by calling exp(gamma*logZ) conjugate. Learned finite-step Langevin chains are not certified target-invariant or universally transferable by the reported experiments. Entropy-tempering requires a normalizable tempered target and meaningful entropy.
• 2016 v1/v2 differ in reported CIFAR values and caption gamma, not just typography; keep versions separate. UAI proceedings and later arXiv1707.06626v2 differ in bibliography (Snelson removed, Pu added), and only the latter downloaded artifact contains Appendices A–C. Empirical comparisons, image quality and speed claims are source-reported and not independently reproduced. No code or training was run.