Home / Curated / Algorithms / Partially noncentered GLMM variational inference

Partially noncentered GLMM variational inference

Algorithms AIMech:PartiallyNoncenteredGLMMVariationalInference DRAFT

A Bayesian generalized linear mixed-model inference procedure that uses cluster-specific partial noncentering to define independent Gaussian fixed-effect and transformed random-effect blocks, factors an inverse-Wishart random-effect covariance, and alternates nonconjugate Gaussian message proposals with an exact covariance coordinate.

Identity Notes & Concept Scope No normalized identity match across all 442 pinned AIO IDs, names and scoped synonyms and 554 maintained IDs, names and aliases. Fixed-string searches of the recovery, old discovery and original Documents checkouts included ignored and hidden files and returned 11, 0 and 0 lines, all 11 read; these are prior source leads, not existing identities. All 28 semantic AIO rows and 99 native names/definitions were assessed. Eight full annotated AIO CLASS nodes with incident edges/generated YAML and six full native neighboring records were inspected. This descriptive local name denotes Tan and Nott Algorithm 3: a cluster-specific partial noncentering transformation, a block Gaussian/inverse-Wishart family in those coordinates, and nonconjugate variational message updates for a Bayesian GLMM. Generic structured/product mean field and NCVMP do not specify this coordinate-induced family. The full joint coefficient Gaussian and independent variance blocks of Gaussian loss-smoothed mixed-model inference differ from this transformed cluster factorization and shared covariance factor. PX-EM expands a likelihood-fitting complete-data model; VEM learns point model parameters. Neither identifies this fully Bayesian posterior procedure. The absence assessment is bounded to this snapshot and inspected corpus.

Claims & Grounded Evidence

7 Claims

Scope #model-and-coordinates

For independent clusters, use Bernoulli observations with logit link or Poisson observations with log link and known positive exposures. Let the predictor be eta_i=X_i*beta+Z_i*u_i, with u_i|D~N(0,D), beta~N(0,Sigma0) and D~IW(nu,S); require positive-definite Sigma0,S and nu>r-1 for r-dimensional random effects. Arrange the shared fixed/random design and cluster-constant covariates so X_i*beta=Z_i*C_i*beta_RG1+X_i_G2*beta_G2. For a specified r-by-r W_i set B_i=[(I-W_i)*C_i,0], a_i=u_i+B_i*beta and V_i=X_i-Z_i*B_i. Then eta_i=V_i*beta+Z_i*a_i and a_i|beta,D~N(B_i*beta,D). This invertible unit-Jacobian change preserves the model; W_i=0 and W_i=I give the source centered and noncentered endpoints. General W_i is a matrix and need not be symmetric or entrywise between zero and one.

supports Source: Variational Inference for Generalized Linear Mixed Models Using Partially Noncentered Parametrizations · Locator: Sections 2–3, Equations 4, 6–7, PDF pp. 4–5

Mechanism #coordinate-dependent-family

At fixed W_i approximate the posterior by q(beta)*q(D)*product_i q(a_i), with full Gaussian covariance inside each beta and a_i block and q(D)=IW(nuq,Sq). This removes dependence across those transformed blocks and with D, not all dependence among the original effects. With q means m_beta,m_i and covariances Q_beta,Q_i, pulling back gives Cov(u_i,beta)=-B_i*Q_beta, Var(u_i)=Q_i+B_i*Q_beta*B_i^T and Cov(u_i,u_j)=B_i*Q_beta*B_j^T for i!=j. These identities follow from u_i=a_i-B_i*beta and the declared independent blocks. Changing W_i generally changes the approximating family in original coordinates. The objective is the expected log joint plus entropy, a lower bound on log model evidence under proper priors and finite expectations.

supports Source: Variational Inference for Generalized Linear Mixed Models Using Partially Noncentered Parametrizations · Locator: Section 3 coordinate transformation, PDF pp. 4–5; Section 4.3, Equation 16 and Algorithm 3, PDF pp. 7–8; Equation 2, PDF p. 3; curator covariance pullback

Mechanism #curvature-noncentering

The source chooses W_i=(I_f_i+Dstar^-1)^-1*Dstar^-1 using positive-definite Dstar and likelihood curvature I_f_i at selected predictors. Poisson curvature is Z_i^T*diag(exposure_i*exp(eta_i))*Z_i; Bernoulli-logit curvature is Z_i^T*diag(s_i*(1-s_i))*Z_i with s_i=sigmoid(eta_i). The Poisson observed-count replacement is a heuristic proxy. W_i can be fixed after initialization or optionally recomputed from current means and Dstar=Sq/(nuq-r-1), which requires nuq>r+1. Matrix multiplication order matters. This Gaussian-motivated choice does not guarantee exact GLMM factorization, optimal W_i or monotone improvement when W_i is changed.

supports Source: Variational Inference for Generalized Linear Mixed Models Using Partially Noncentered Parametrizations · Locator: Section 3.1, Equations 8–10, PDF p. 5; Algorithm 3 step 1, PDF p. 8

Mechanism #predictor-moments

Under the current Gaussian blocks, each predictor is normal with mean m_eta_i=V_i*m_beta+Z_i*m_i and variance vector v_i=diag(V_i*Q_beta*V_i^T+Z_i*Q_i*Z_i^T). For Poisson, both expected first and second derivatives of its cumulant are G_i=F_i=exposure_i*exp(m_eta_i+v_i/2), elementwise. For logistic, G_ij=E[sigmoid(m_eta_ij+sqrt(v_ij)*Z0)] and F_ij=E[sigmoid(t)*(1-sigmoid(t))], where t is that same scalar normal predictor. The logistic expectations reduce to one-dimensional Gaussian integrals. The source approximates softplus and its first two derivatives by adaptive Gauss-Hermite quadrature; these exact integral identities do not certify finite adaptive-quadrature error or that derivatives of a discrete approximation coincide.

supports Source: Variational Inference for Generalized Linear Mixed Models Using Partially Noncentered Parametrizations · Locator: Section 4.3 definitions of F, G and B derivatives, PDF p. 8; Appendix B, PDF pp. 19–20

Mechanism #gaussian-block-proposals

Hold W fixed during a Gaussian block update and let A=E_q[D^-1]=nuq*Sq^-1. At the current beta block set P_beta=Sigma0^-1+sum_i B_i^T*A*B_i+sum_i V_i^T*diag(F_i)*V_i and g_beta=-Sigma0^-1*m_beta+sum_i B_i^T*A*(m_i-B_i*m_beta)+sum_i V_i^T*(y_i-G_i). Propose Q_beta_new=P_beta^-1 and m_beta_new=m_beta+Q_beta_new*g_beta, with F,G and derivatives evaluated at the same old beta state. Then update each cluster using current neighboring blocks: P_i=A+Z_i^T*diag(F_i)*Z_i, g_i=-A*(m_i-B_i*m_beta)+Z_i^T*(y_i-G_i), Q_i_new=P_i^-1 and m_i_new=m_i+Q_i_new*g_i. Recompute predictor moments after changed blocks. Proper priors and exact nonnegative curvature make these precisions positive definite; these are Gaussian NCVMP proposals, not exact nonconjugate coordinate maxima or general ascent/convergence guarantees.

supports Source: Variational Inference for Generalized Linear Mixed Models Using Partially Noncentered Parametrizations · Locator: Section 4.2 Equation 15 and convergence warning, PDF p. 7; Algorithm 3 steps 2–3, PDF p. 8; derivative derivation, PDF pp. 8–9

Mechanism #inverse-wishart-coordinate

After the Gaussian block proposals, the exact conjugate covariance coordinate is nuq=nu+n and Sq=S+sum_i[(m_i-B_i*m_beta)*(m_i-B_i*m_beta)^T+Q_i+B_i*Q_beta*B_i^T], where n is the number of clusters. This follows from the expected original random-effect outer products, including uncertainty in beta. In the convention used here the inverse-Wishart density is proportional to |D|^(-(nuq+r+1)/2)*exp(-tr(Sq*D^-1)/2); E[D^-1]=nuq*Sq^-1. Proper factor normalization requires nuq>r-1 and Sq positive definite, while its mean additionally needs nuq>r+1. Omitting the beta-covariance term would not be this coordinate update.

supports Source: Variational Inference for Generalized Linear Mixed Models Using Partially Noncentered Parametrizations · Locator: Algorithm 3 step 4 and conjugate argument, PDF p. 8; Appendix A inverse-Wishart expectation, PDF p. 18

Scope #gaussian-special-case

In the motivating Gaussian linear mixed model with shared fixed/random design X_i, known positive residual variance sigma2 and known positive-definite D, take W_i=(X_i^T*X_i/sigma2+D^-1)^-1*D^-1. Then D^-1*(I-W_i)=(X_i^T*X_i/sigma2)*W_i, cancelling the beta–a_i posterior precision blocks. When the posterior is proper, the transformed exact Gaussian posterior factorizes into the chosen beta and cluster blocks, so one coordinate cycle recovers it. The source uses a flat beta prior; a proper Gaussian beta prior preserves this cancellation by adding only to the beta precision block, as checked in the curator example. This special-case exactness does not extend automatically to unknown D or Bernoulli/Poisson likelihoods, and flat-prior model evidence has an arbitrary normalization.

supports Source: Variational Inference for Generalized Linear Mixed Models Using Partially Noncentered Parametrizations · Locator: Section 1.1 and Algorithm 1, PDF pp. 2–3; curator noncommuting-matrix calculation

⚠️ Curated Limitations & Negative Results

Partial noncentering changes a restricted variational family, not the underlying model. The model-coordinate map is exact, but the factorized posterior approximation remains restricted. Its induced original-effect covariance does not represent an arbitrary joint Gaussian, and dependence with D is still omitted.
The optional W update is a curvature heuristic, not an exact ELBO coordinate maximization. Its inverse-Wishart mean exists only for nuq>r+1; with the source choice nu=r this requires more than one cluster. The empirical prior scale S=r*Rhat and penalized quasi-likelihood initialization require nonsingular information; near-zero random-effect variance or unstable initial covariance can cause numerical problems.
The source itself warns that nonconjugate message proposals need not increase the lower bound or converge. Positive precision with exact Bernoulli/Poisson curvature is not an ascent theorem. Small relative bound changes do not establish a global optimum, posterior accuracy or uniqueness.
Appendix A’s compact final bound is stated for the parameters after their updates, including the inverse-Wishart coordinate; it is not automatically valid for arbitrary stale-scale trial states. Replacing log evidence by an attained ELBO does not guarantee correct Bayes-factor ordering or model posterior probabilities.
All 21 primary reprint pages, including experiments, tables, figures, appendices and references, were read. Numerical checks use independent finite matrix arithmetic, supplied Gaussian states and Simpson quadrature; they do not reproduce the source adaptive ten-node Gauss-Hermite code, convergence or benchmarks. No author implementation or GLMM fitting was executed, and no quantitative runtime/accuracy superiority or clinical conclusion is asserted. DRAFT remains.

Artificial Intelligence Ontology (AIO) Grounding

Mapped AIO Classes

AIO Target Class	Mapping Relation	Mapping Rationale
`AIO:RandomEffectsModel` Random Effects Model	related_match	This procedure performs Bayesian approximate inference for clustered generalized linear models with Gaussian random effects and an unknown shared random-effect covariance. The AIO class describes the random-effects modeling context, not the coordinate transformation, restricted variational family or inference schedule. This algorithm-to-model-family relation asserts neither equivalence nor an is-a link.

Evidence Sources & Primary Literature

1 Source

Source ID	Publication / Document	Type / Access	Reference & Checksum
`tan2013`	Variational Inference for Generalized Linear Mixed Models Using Partially Noncentered Parametrizations Linda S. L. Tan, David J. Nott	primary paper full_text	ID: https://doi.org/10.1214/13-STS418 sha256:32f56daf18e0…

Curation Provenance

Curator: Codex

Method: ai_assisted

Model: GPT-6

Created: 2026-10-13T00:00:33Z

Updated: 2026-10-13T00:00:33Z

Source File: data/curated/PartiallyNoncenteredGLMMVariationalInference.yaml