Uniformly valid post-model-selection confidence intervals via consistent variance overestimation
Mathematical Methods AIMech:UniformlyValidPostModelSelectionConfidenceIntervals DRAFT
Aliases: Bachoc-Preinerstorfer-Steinberger uniformly valid post-model-selection confidence intervals, Asymptotically uniformly valid PoSI confidence intervals under model misspecification, Consistent variance overestimation post-selection confidence intervals (CI^{oest}, CI^{lm}, CI^{hlm}, CI^{bin}), Heteroskedastic Eicker and Fahrmeir binary regression post-model-selection confidence intervals
A general post-model-selection inference framework and family of confidence intervals CI^{(j)}_{1-alpha, M_hat_n}(y) = theta_hat^{(j)}_{M_hat_n, n}(y) +- sqrt(sigma_hat^2_{rho(M_hat_n)+j, n}(y)) * K_hat_n(y) (Bachoc, Preinerstorfer, & Steinberger 2020, Equation 2.6) for stacked asymptotically linear estimators theta_hat_n(y) - theta^*_n = sum_{i=1}^n g_{i,n}(y_i) + Delta_n(y) in R^k (k = sum_{j=1}^d m(M_{j,n})) satisfying Condition 1, guaranteeing asymptotic coverage liminf_{n -> infty} inf_{P_n in P_n} inf_{M_hat_n} P_n(theta^{*(j)}_{M_hat_n, n} in CI^{(j)}_{1-alpha, M_hat_n}(y)) >= 1 - alpha uniformly over both arbitrary data-driven model selectors M_hat_n and rich classes of possibly misspecified triangular-array distributions P_n (Equations 1.1-1.3, Lemma 2.2, Theorem 2.4, Theorem 2.6). Because uniformly consistent estimation of the true error variance sigma_n^2 is impossible under unbounded model misspecification (Proposition 3.5), the framework replaces consistent variance estimators with consistent variance overestimators nu_hat^2_{j,n}(y) = sum_{i=1}^n [g_tilde^{(j)}_{i,n}(y)]^2 constructed from observable score summands g_tilde^{(j)}_{i,n}(y) = g_hat^{(j)}_{i,n}(y) + a^{(j)}_{i,n} that include unobservable deterministic misspecification shifts a^{(j)}_{i,n}, satisfying P_n(sqrt([V_n(r_n)]_j / nu_hat^2_{j,n}) >= 1 + varepsilon) -> 0 for all misspecification sequences and adapting to exact consistency when misspecification is negligible (Proposition 2.7, Remark 2.8). Paired with either the known-correlation PoSI constants K_{1-alpha}(corr(Gamma_n)) and K_{1-alpha}(corr(Xi_n)) in homoskedastic linear regression (Theorems 3.2-3.3, Proposition 3.4) or the distribution-free rank-adaptive bound B_alpha(rank(Gamma), k) = min_{lambda in (0, 1)} sqrt(q_{1 - lambda*alpha, rank(Gamma)} + 2 log(k / ((1 - lambda)*alpha))) (Lemma 2.9), the framework yields uniformly valid post-selection confidence intervals for heteroskedastic linear regression with Eicker sandwich variances and B_alpha(min(k, p), k) (Theorem 3.6) and for misspecified binary regression with joint link and regressor selection via Fahrmeir quasi-MLE sandwich variances and B_alpha(min(k, n), k) (Theorem 3.11), which reduces to B_alpha(min(k, p), k) under the canonical logistic link (Remark 3.12, Corollary D.3).
Identity Notes & Concept Scope
Distinct from AIMech:ValidPostSelectionInferencePoSIConstant (Berk, Brown, Buja, Zhang, & Zhao 2013, The Annals of Statistics 41(2):802-837, arXiv:1206.6451), which defines the fixed-design Gaussian homoskedastic linear regression PoSI constant K(X, M_all, alpha, r) assuming an independent homoskedastic error variance estimator sigma_hat^2 ~ (sigma^2 / r) chi^2_r exists. Distinct from AIMech:RestrictedIsometryPostSelectionInferenceConstantBounds (Bachoc, Blanchard, & Neuvial 2018, Electronic Journal of Statistics 12(2):3736-3757, arXiv:1804.07566v2), which bounds the fixed-design Gaussian PoSI constant under s-sparse Restricted Isometry Property assumptions. Distinct from AIMech:PostModelSelectionPredictorPoSIConfidenceIntervals (Bachoc, Leeb, & Pötscher 2019, The Annals of Statistics 47(3):1475-1504, arXiv:1412.4605v3), which constructs post-selection confidence intervals for linear predictors x_0'[M_hat] beta^*_{M_hat} under full or active-only observation of a new covariate vector x_0 in homoskedastic linear regression. Bachoc, Preinerstorfer, & Steinberger (2020, The Annals of Statistics 48(1):440-463, arXiv:1611.01043v4, DOI:10.1214/19-AOS1815) establish a general triangular-array asymptotic linearity framework (Condition 1, Lemma 2.2) across arbitrary model families, prove that uniformly consistent estimation of sigma_n^2 is impossible under unbounded misspecification (Proposition 3.5), introduce consistent variance overestimation nu_hat^2_{j,n} = sum_{i=1}^n [g_tilde^{(j)}_{i,n}]^2 (Theorem 2.6, Proposition 2.7), and construct uniformly valid post-selection confidence intervals for homoskedastic linear models (Theorems 3.2-3.3), heteroskedastic linear models via Eicker sandwich estimators (Theorem 3.6), and misspecified binary regression with joint link and regressor selection via Fahrmeir quasi-MLE sandwich estimators (Lemmas 3.9-3.10, Theorem 3.11, Remark 3.12, Corollary D.3).
Claims & Grounded Evidence
4 Claims
Mechanism
#asymptotic-linearity-clt-and-consistent-variance-overestimation
For a triangular array of independent observations y_i in Y_{i,n} (i = 1, ..., n) with joint distribution P_n in P_n and stacked candidate parameter estimators theta_hat_n(y) = (theta_hat'_{M_{1,n}, n}(y), ..., theta_hat'_{M_{d,n}, n}(y))' in R^k (k = sum_{j=1}^d m(M_{j,n})), Condition 1 (Section 2, page 8) requires theta_hat_n(y) - theta^*_n = r_n(y) + Delta_n(y) with r_n(y) = sum_{i=1}^n g_{i,n}(y_i), E_n(g_{i,n}(y_i)) = 0, uniform componentwise Lindeberg condition lim_{n -> infty} sup_{P_n in P_n} max_{j=1,...,k} [V_n(r_n)]_j^{-1} sum_{i=1}^n E_n([g^{(j)}_{i,n}]^2 1_{{|g^{(j)}_{i,n}| >= eps sqrt([V_n(r_n)]_j)}}) = 0, and uniformly negligible remainder diag(V_n(r_n))^{dagger/2} Delta_n ->^{P_n} 0. Using Pollak (1972) and Raikov's (1938) theorem, Lemma 2.2 and Lemma B.1 (pages 9-10, 38-39) prove that d_w(P_n o [diag(V_n(r_n))^{dagger/2}(theta_hat_n - theta^*_n)], N(0, corr(V_n(r_n)))) -> 0 and d_w(P_n o [diag(S_n)^{dagger/2}(theta_hat_n - theta^*_n)], N(0, corr(S_n))) ->^{P_n} 0 for S_n = sum_{i=1}^n g_{i,n}(y_i) g'_{i,n}(y_i). When model misspecification shifts the observable score summands by non-stochastic biases a^{(j)}_{i,n} so that g_tilde^{(j)}_{i,n}(y) = g_hat^{(j)}_{i,n}(y) + a^{(j)}_{i,n}, Proposition 2.7 and Lemma B.4(3) (pages 14-15, 44-46) prove that the uncentered empirical second moment nu_hat^2_{j,n}(y) = sum_{i=1}^n [g_tilde^{(j)}_{i,n}(y)]^2 satisfies P_n(sqrt([V_n(r_n)]_j / nu_hat^2_{j,n}) >= 1 + varepsilon) -> 0 (consistent overestimation, reducing to exact consistency when [V_n(r_n)]_j^{-1} sum_{i=1}^n [a^{(j)}_{i,n}]^2 -> 0 by Remark 2.8), so Theorem 2.6 guarantees uniform asymptotic coverage >= 1 - alpha for CI^{(j), oest}_{1-alpha, M_hat_n} (Equation 2.7), with Lemma 2.9 (page 16) bounding K_{1-alpha}(Gamma) <= B_alpha(rank(Gamma), k) = min_{lambda in (0, 1)} sqrt(q_{1 - lambda*alpha, rank(Gamma)} + 2 log(k / ((1 - lambda)*alpha))).
Mechanism
#homoskedastic-linear-model-adaptation-and-uniform-impossibility
In the misspecified homoskedastic linear regression model y = mu_n + u_n with E_n(u_n) = 0, V_n(u_n) = sigma_n^2 I_n, and candidate regressor subsets M in I under Condition X1 (Section 3.1, pages 16-21, Equations 3.1-3.2), the stacked OLS influence matrix G_n in R^{k x n} has known deterministic correlation matrix corr(Gamma_n) = corr(G_n G'_n), and Proposition 3.4 (page 20, proved in Appendix C.3, page 50) shows that the submodel residual variance estimator sigma_hat^2_{M,n}(y) = (n - |M|)^{-1} y'(I_n - P_{X_n[M]}) y satisfies sigma_hat^2_{M,n} / sigma_n^2 = (1 + o_{P_n}(1))(1 + ||(I_n - P_{X_n[M]}) mu_n / sigma_n||^2 / (n - |M|)) uniformly over P_n^{(lm)}(delta, tau). Consequently, sigma_hat^2_{M,n} consistently overestimates sigma_n^2 under misspecification and adapts to exact consistency whenever mu_n in span(X_n[M]), so Theorem 3.2 (page 18) guarantees uniform asymptotic coverage >= 1 - alpha for CI^{(j), lm}_{1-alpha, M} = beta_hat^{(j)}_{M,n} +- sqrt(sigma_hat^2_{M,n} [(X_n[M]' X_n[M])^{-1}]_j) K_{1-alpha}(corr(Gamma_n)) (Equation 3.3), and Theorem 3.3 (page 19) replaces Gamma_n by the smaller d x d covariance matrix Xi_n when covering a single data-selected coefficient j_hat_n in {1, ..., |M_hat_n|}. Conversely, Proposition 3.5 (page 21, proved in Appendix C.4, pages 50-51) proves that whenever limsup_{n -> infty} rank(X_n)/n < 1 and tau > 1, no sequence of estimators tilde_sigma_n^2 can estimate sigma_n^2 uniformly consistently over P_n^{(lm)}(delta, tau).
Mechanism
#heteroskedastic-eicker-sandwich-post-selection-confidence-intervals
In misspecified heteroskedastic linear regression P_n^{(het)}(delta, tau) with independent errors u_{i,n} of heterogeneous variances sigma^2_{i,n} satisfying max_{i,j} (sum_{l=1}^n [G^{(j)}_{l,n}]^2 sigma^2_{l,n})^{-1} [G^{(j)}_{i,n}]^2 sigma^2_{i,n} <= tau / n, Theorem 3.6 (Section 3.2, pages 21-22, proved in Appendix C.5, page 51) proves that combining the Eicker (1963) sandwich variance estimator sigma_hat^2_{j, M, n}(y) = sum_{i=1}^n (e'_j (X_n[M]' X_n[M])^{-1} X_{i,n}[M]')^2 u_hat^2_{i, M, n}(y) (where u_hat_{M,n}(y) = (I_n - P_{X_n[M]}) y) with the rank-bounded PoSI constant B_alpha(min(k, p), k) (since rank(V_n(r_n)) = rank(G_n diag(sigma^2_{1,n}, ..., sigma^2_{n,n}) G'_n) <= min(k, p)) yields confidence intervals CI^{(j), hlm}_{1-alpha, M}(y) = beta_hat^{(j)}_{M,n}(y) +- sqrt(sigma_hat^2_{j, M, n}(y)) B_alpha(min(k, p), k) (Equation 3.4) with uniform asymptotic coverage >= 1 - alpha over P_n^{(het)}(delta, tau) and all model selectors M_hat_n.
Mechanism
#misspecified-binary-regression-fahrmeir-sandwich-and-canonical-link-reduction
In misspecified binary regression P_n^{(bin)}(tau) where independent binary outcomes y_i in {0, 1} have arbitrary success probabilities p_{i,n} = P_n(y_i = 1) in [1/tau, 1 - 1/tau] and working models M = (M, P_{M, h}) jointly select both a regressor subset M in I and a link function h in H under Conditions X2 and H (Section 3.3, pages 22-28), Lemmas 3.9-3.10 (pages 25-26, proved in Appendix D.4-D.5, pages 53-58) prove unique existence of the Kullback-Leibler projection target beta^*_{M,n} = argmin_{beta in R^{|M|}} KL(P_n, P_{M,h,beta}) solving sum_{i=1}^n [p_{i,n} phi_dot_1(X_{i,n}[M] beta) + (1 - p_{i,n}) phi_dot_2(X_{i,n}[M] beta)] X'_{i,n}[M] = 0 (Equation 3.5) and uniform consistency of the quasi-MLE beta_hat_{M,n}. Theorem 3.11 (page 27, proved in Appendix D.6-D.8, pages 58-63) proves that pairing the Fahrmeir (1990) sandwich variance estimator S_tilde_{M,n}(y) = J_hat^{-1}_{M,n}(beta_hat_{M,n}) V_hat_{M,n}(s_n) J_hat^{-1}_{M,n}(beta_hat_{M,n}) (Equation 3.6) with B_alpha(min(k, n), k) yields confidence intervals CI^{(j), bin}_{1-alpha, M}(y) = beta_hat^{(j)}_{M,n}(y) +- sqrt([S_tilde_{M,n}(y)]_j) B_alpha(min(k, n), k) (Equation 3.7) with uniform asymptotic coverage >= 1 - alpha over P_n^{(bin)}(tau). Moreover, Remark 3.12 (page 27) and Corollary D.3 (page 53) prove that for the canonical logistic link h^{(c)}(gamma) = e^gamma / (1 + e^gamma), phi_dot_1(gamma) - phi_dot_2(gamma) === 1, so the score vectors lie in the p-dimensional column space of X'_n and rank(V_n(r_n)) <= min(k, p), reducing the binary regression PoSI constant from B_alpha(min(k, n), k) to B_alpha(min(k, p), k).
⚠️ Curated Limitations & Negative Results
- The uniform multivariate central limit theorem in Lemma 2.2 treats the total number of stacked candidate parameters k = sum_{j=1}^d m(M_{j,n}) and the regressor dimension p as fixed independent of sample size n (Section 2, Footnote 6, page 8).
- Because uniformly consistent estimation of the true error variance sigma_n^2 is impossible under unbounded model misspecification when tau > 1 (Proposition 3.5), the consistent variance overestimator nu_hat^2_{j,n} = sum_{i=1}^n [g_tilde^{(j)}_{i,n}]^2 intentionally overestimates [V_n(r_n)]_j when misspecification biases a^{(j)}_{i,n} are large, yielding conservative coverage for severely misspecified working models while adapting to exact consistency when the selected model is well-specified (Proposition 2.7, Remark 2.8, Proposition 3.4).
- Four minor typographical notes in arXiv:1611.01043v4 were identified and verified during full-text inspection: (1) Section 1.1 (page 4, line 11) prints 'sample spitting procedure' for 'sample splitting procedure'; (2) Remark 3.13 (page 28, line 16) prints 'Conditon 1' for 'Condition 1'; (3) Sections 4.1-4.2 (pages 29-30) refer to 'Table 3' and 'Table 4' (the Appendix A table numbers on pages 36-37) when discussing what are labeled 'Table 1' and 'Table 2' in Section 4; and (4) Appendix D.8 (page 61, line 1) prints '\beta^*_{\mathbb{M}.n}' with a period instead of a comma in the subscript.
- All 70 pages of the full-text PDF of Bachoc, Preinerstorfer, & Steinberger (2020, The Annals of Statistics 48(1):440-463, arXiv:1611.01043v4, DOI:10.1214/19-AOS1815) were read. Constructed numerical checks in research/discovery/uniformly-valid-post-model-selection-confidence-intervals-boundaries.py verify (1) Proposition 2.7 consistent variance overestimation under misspecification and Remark 2.8 exact consistency under well-specification, (2) Proposition 3.4 homoskedastic variance adaptation, Theorem 3.3 vs. Theorem 3.2 PoSI constants, and Proposition 3.5's impossibility mixture, (3) Theorem 3.6 heteroskedastic Eicker sandwich PoSI coverage with Lemma 2.9's B_alpha(min(k, p), k) bound under significance hunting, and (4) Remark 3.12 / Corollary D.3 canonical logistic link score rank reduction from B_alpha(min(k, n), k) to B_alpha(min(k, p), k).
Artificial Intelligence Ontology (AIO) Grounding
Mapped AIO Classes
| AIO Target Class | Mapping Relation | Mapping Rationale |
|---|---|---|
AIO:LinearRegression
Linear Regression
|
related_match | Instantiates uniformly valid post-model-selection confidence intervals for misspecified homoskedastic (Theorems 3.2-3.3) and heteroskedastic (Theorem 3.6) linear regression models (AIO:LinearRegression). |
AIO:LogisticRegression
Logistic Regression
|
related_match | Establishes uniformly valid post-model-selection confidence intervals for misspecified binary regression with joint link and regressor selection (Theorem 3.11) and proves rank reduction of the PoSI constant for canonical-link logistic regression (Remark 3.12, Corollary D.3; AIO:LogisticRegression). |
AIO:DataDredgingBias
Data Dredging Bias
|
related_match | Guarantees asymptotic coverage >= 1 - alpha uniformly over arbitrary data-driven model-selection procedures M_hat_n including significance hunting, preventing post-selection data-snooping invalidation (AIO:DataDredgingBias). |
Evidence Sources & Primary Literature
3 Sources| Source ID | Publication / Document | Type / Access | Reference & Checksum |
|---|---|---|---|
bps2020-pdf
|
François Bachoc, David Preinerstorfer, Lukas Steinberger
|
primary paper
full_text
|
ID: https://doi.org/10.1214/19-AOS1815
sha256:8cfe880631d0…
|
bps2020-crossref
|
Uniformly valid confidence intervals post-model-selection (The Annals of Statistics Crossref Record)
François Bachoc, David Preinerstorfer, Lukas Steinberger
|
primary paper
abstract_only
|
ID: https://doi.org/10.1214/19-AOS1815
sha256:811c958acffd…
|
bps2020-abs
|
François Bachoc, David Preinerstorfer, Lukas Steinberger
|
primary paper
abstract_only
|
ID: https://doi.org/10.48550/arXiv.1611.01043
sha256:60d4499beb32…
|
Ontology & Grounding Topology
Open in interactive network graph →Curation Provenance
Curator: Antigravity
Method: ai_assisted
Model: Gemini 3
Created: 2026-10-17T17:45:00Z
Updated: 2026-10-17T17:45:00Z
Source File:
data/curated/UniformlyValidPostModelSelectionConfidenceIntervals.yaml
· Schema:
curated_concept.yaml
Explore across X-Mech
Related computational & biological Mechs:
Embedding Map
Network Graph
ProteinTraitsMech (ESM-2 pLM)
DUFMech (Domain pLM)
TraitMech
CultureMech
All 14 X-Mech Repositories