Unified assumption-lean moving-target linear regression and score bootstrap
Mathematical Methods AIMech:UnifiedAssumptionLeanMovingTargetLinearRegressionAndScoreBootstrap DRAFT
Aliases: Model-free study of ordinary least squares linear regression (Kuchibhotla, Brown, & Buja 2018), Unified independent non-identically distributed moving-target OLS framework, Multiplier score bootstrap and m-of-n resampling score bootstrap for ordinary least squares, Deterministic master inequality and semiparametric efficiency for moving-target OLS
An assumption-lean statistical estimation and inference methodology for ordinary least squares (OLS) linear regression that starts with the empirical OLS estimator beta_hat_n = Sigma_hat_n^{-1} Gamma_hat_n (where Sigma_hat_n := 1/n sum_{i=1}^n X_i X_i^T and Gamma_hat_n := 1/n sum_{i=1}^n X_i Y_i) and defines its sample-size-dependent moving target of estimation beta_n := Sigma_n^{-1} Gamma_n = argmin_{theta in R^p} 1/n sum_{i=1}^n E[(Y_i - X_i^T theta)^2] (where Sigma_n := 1/n sum_{i=1}^n E[X_i X_i^T] and Gamma_n := 1/n sum_{i=1}^n E[X_i Y_i]; Definition 2.1, Equations 9-14) across a unified framework of independent, possibly non-identically distributed (i.n.i.d.) observation vectors (X_i^T, Y_i)^T in R^{p+1} encompassing both fixed covariates and random covariates (Kuchibhotla, Brown, & Buja 2018). Because the population normal equation 1/n sum_{i=1}^n E[X_i(Y_i - X_i^T beta_n)] = 0 holds on average while individual score expectations E[X_i(Y_i - X_i^T beta_n)] are generally non-zero under fixed or non-identically distributed covariates with nonlinearity, the centered score vectors S_i := X_i(Y_i - X_i^T beta_n) - E[X_i(Y_i - X_i^T beta_n)] have true covariance K_n = 1/n sum_{i=1}^n E[S_i S_i^T] satisfying Bentkus's multidimensional Berry-Esseen bound over convex sets C_p subseteq R^p (Theorem 2.1, Theorem 5.1, Equations 16-17), while the uncentered second-moment matrix K_n^* := 1/n sum_{i=1}^n E[X_i X_i^T (Y_i - X_i^T beta_n)^2] dominates K_n in Loewner order (K_n <= K_n^*). Pairing the empirical plug-in matrix K_check_n := 1/n sum_{i=1}^n X_i X_i^T (Y_i - X_i^T beta_hat_n)^2 with Anderson's (1955) Lemma (Lemma 6.2), the methodology constructs asymptotically conservative confidence regions and simultaneous tests via the Eicker-Huber-White sandwich estimator Sigma_hat_n^{-1} K_check_n Sigma_hat_n^{-1} (Equation 18) and matrix-inversion-free multiplier and m-of-n resampling score bootstraps on the estimated score vectors S_hat_i := X_i(Y_i - X_i^T beta_hat_n) (Sections 6.2-7, Equations 20-29), backed by an assumption-free deterministic master inequality (Theorem 8.1, Equations 30-32) and semiparametric efficiency of beta_hat_n for beta_n (Appendix A, Lemma A.1, Equations 33-36).
Identity Notes & Concept Scope
Distinct from all 442 pinned AIO classes (including AIO:LinearRegression) and all 610 maintained AIMech curated concepts. Specifically distinct from AIMech:ModelRobustNonlinearityConsistentSandwichAndRAVSpecificationTest (Buja et al. 2019 Part I, Statistical Science 34(4):523-544, arXiv:1404.1578v4, DOI:10.1214/18-STS693; restricted to i.i.d. random-design regression, Conditional Lock-In Theorem 3.1 for M_Lin and M_Hom, and the Ratio of Asymptotic Variances test statistic RAV_j) and AIMech:ModelFreeRegressionFunctionalsReweightingDiagnosticAndXYBootstrapLimit (Buja et al. 2019 Part II, Statistical Science 34(4):545-565, arXiv:1612.03257v2, DOI:10.1214/18-STS694; general optimization, proper scoring rule, and estimating-equation functionals on i.i.d. P_X x P_{Y|X}, Gaussian decile reweighting diagnostic trace, partial influence function IF(x_vec), and M-of-N XY-bootstrap limit). Kuchibhotla, Brown, & Buja (2018, arXiv:1809.10538v1) formulate the unified independent non-identically distributed (i.n.i.d.) moving-target OLS framework beta_n = Sigma_n^{-1} Gamma_n bridging fixed and random covariates (Definition 2.1, Theorem 2.1, Theorem 5.1), prove the impossibility of consistent variance estimation under arbitrary unknown means (Lemma 6.1) alongside Loewner-order conservativeness K_n <= K_n^* and Anderson's Lemma (Lemma 6.2), introduce the multiplier score bootstrap T_n^* = n^{-1/2} sum_{i=1}^n W_i S_hat_i and m-of-n resampling score bootstrap T_m^star = m^{-1/2} sum_{j=1}^m S_hat_{I_j} that avoid solving singular resampled linear systems (Sections 6.3-6.4), and prove the deterministic master inequality for OLS (Theorem 8.1) and semiparametric efficiency of beta_hat_n for beta_n under i.n.i.d. product measures (Appendix A, Lemma A.1).
Claims & Grounded Evidence
4 Claims
Mechanism
#unified-inid-moving-target-ols-and-multidimensional-berry-esseen-bound
Under the unified independent, possibly non-identically distributed (i.n.i.d.) framework for (X_i^T, Y_i)^T in R^{p+1} encompassing fixed covariates (point masses at x_i, Section 3) and i.i.d. random covariates (Section 4), the moving target beta_n := Sigma_n^{-1} Gamma_n = argmin_{theta in R^p} 1/n sum_{i=1}^n E[(Y_i - X_i^T theta)^2] (Definition 2.1, Equations 9-14) satisfies the average population normal equation 1/n sum_{i=1}^n E[X_i(Y_i - X_i^T beta_n)] = 0 while the individual score expectations E[X_i(Y_i - X_i^T beta_n)] vanish under i.i.d. observations but are generally non-zero under fixed or non-identically distributed covariates with nonlinearity (Section 5, p. 11). Defining the centered score vectors S_i := X_i(Y_i - X_i^T beta_n) - E[X_i(Y_i - X_i^T beta_n)] (Equation 16) with covariance K_n = 1/n sum_{i=1}^n E[S_i S_i^T], Bentkus's (2004) multidimensional Berry-Esseen bound over all convex sets C_p subseteq R^p yields sup_{A in C_p} |P(n^{1/2} Sigma_hat_n (beta_hat_n - beta_n) in A) - P(N(0, K_n) in A)| <= C p^{1/4} n^{-1/2} alpha_n where alpha_n := 1/n sum_{i=1}^n E[||K_n^{-1/2} S_i||_2^3] (Theorem 2.1, Equation 17), proving n^{1/2} K_n^{-1/2} Sigma_hat_n (beta_hat_n - beta_n) ->^L N(0, I_p) under uniform sixth moment bounds E[||X_i||_2^6 + Y_i^6] <= B < infty (Theorem 5.1, pp. 12-14).
Mechanism
#variance-estimation-impossibility-loewner-conservativeness-and-anderson-lemma
For independent random variables W_1, ..., W_n with unknown means E[W_i] = mu_i and variances Var(W_i) = sigma_i^2, there does not exist any consistent estimator for the average variance eta_n^2 := 1/n sum_{i=1}^n sigma_i^2 over arbitrary {(mu_i, sigma_i^2)}_{i=1}^n (Section 6.1, Lemma 6.1, p. 16). However, the uncentered second-moment matrix K_n^* := 1/n sum_{i=1}^n E[X_i X_i^T (Y_i - X_i^T beta_n)^2] dominates the true centered score covariance K_n in Loewner order via K_n = K_n^* - 1/n sum_{i=1}^n (E[X_i(Y_i - X_i^T beta_n)])(E[X_i(Y_i - X_i^T beta_n)])^T <= K_n^* (p. 14), the empirical plug-in matrix K_check_n := 1/n sum_{i=1}^n X_i X_i^T (Y_i - X_i^T beta_hat_n)^2 satisfies ||K_check_n - K_n^*||_op = o_p(1) (p. 15), and by Anderson's (1955) Lemma (Lemma 6.2, p. 18), for any centrally symmetric convex set A subseteq R^p, K_n <= K_n^* implies P(N(0, K_n^*) in A) <= P(N(0, K_n) in A), ensuring that confidence regions and coordinate t_j / simultaneous max-|t| tests constructed from the Eicker-Huber-White sandwich estimator Sigma_hat_n^{-1} K_check_n Sigma_hat_n^{-1} (Equation 18) are asymptotically exact under i.i.d. sampling (K_n^* = K_n) and asymptotically conservative across the unified i.n.i.d. and fixed-covariate settings (Section 6.1, Section 7, pp. 14-22).
Mechanism
#multiplier-and-m-of-n-resampling-score-bootstraps-without-matrix-inversion
Because classical pairs bootstrap resampling of observations (X_i, Y_i) requires solving B linear systems and encounters singular resampled Gram matrices with positive probability when covariates have discrete atoms or small sample sizes (Section 6.2, pp. 16-17), Sections 6.3-6.4 (pp. 17-21, Equations 20-29) bootstrap the estimated score vectors S_hat_i := X_i(Y_i - X_i^T beta_hat_n) (which satisfy sum_{i=1}^n S_hat_i = 0 and 1/n sum_{i=1}^n S_hat_i S_hat_i^T = K_check_n after solving the OLS linear system only once) via either the multiplier score bootstrap T_n^* := n^{-1/2} sum_{i=1}^n W_i S_hat_i with i.i.d. multipliers E[W_i] = 0, E[W_i^2] = 1, E[|W_i|^3] < infty (where W_i ~ N(0,1) makes the conditional Berry-Esseen error in Equation 21 identically zero, p. 18) or the m-of-n resampling score bootstrap T_m^star := m^{-1/2} sum_{j=1}^m S_hat_{I_j} (Equation 28). Both score bootstraps converge conditionally to N(0, K_n^*) over all convex sets and yield Glivenko-Cantelli confidence regions R_hat_n(alpha) for T_n = n^{1/2} Sigma_hat_n (beta_hat_n - beta_n) satisfying inf_{alpha in [0,1]} (int_{R_hat_n(alpha)} dP_{T_n} - (1 - alpha)) >= o_p(1) (Equations 24-27).
Mechanism
#deterministic-master-inequality-and-semiparametric-efficiency-of-ols
Section 8 (pp. 23-24, Theorem 8.1, Equations 30-32) proves that for any sample realization satisfying D_{2n}^Sigma := ||Sigma_hat_n - Sigma_n||_op <= Lambda_n / 2 where Lambda_n := lambda_min(Sigma_n) > 0, the OLS estimator beta_hat_n and moving target beta_n obey the assumption-free deterministic master inequalities 1/2 ||Sigma_n^{-1}(Gamma_hat_n - Sigma_hat_n beta_n)||_2 <= ||beta_hat_n - beta_n||_2 <= 2 ||Sigma_n^{-1}(Gamma_hat_n - Sigma_hat_n beta_n)||_2 (Equation 30) and ||beta_hat_n - beta_n - Sigma_n^{-1}(Gamma_hat_n - Sigma_hat_n beta_n)||_2 <= 2 D_{2n}^Sigma ||Sigma_n^{-1}(Gamma_hat_n - Sigma_hat_n beta_n)||_2 / Lambda_n (Equation 31) without any distributional or independence assumptions. Furthermore, Appendix A (pp. 25-31, Lemma A.1, Equations 33-36) proves via least favorable parametric submodels dP_t^{(g_1,...,g_n)}(z_1,...,z_n) = prod_{i=1}^n c(t, g_i) K(t g_i(z_i)) dP_i(z_i) with K(u) = 2(1 + exp(-2u))^{-1} that the efficient influence function for beta_n is psi_tilde_{P_i}(Z_i) = (sum_{i=1}^n E[X_i X_i^T])^{-1} {X_i(Y_i - X_i^T beta_n) - E[X_i(Y_i - X_i^T beta_n)]}, establishing that beta_hat_n is a semiparametrically efficient regular estimator of beta_n.
⚠️ Curated Limitations & Negative Results
- Under non-identically distributed or fixed covariates with nonlinearity where the individual score expectations E[X_i(Y_i - X_i^T beta_n)] are non-zero, Lemma 6.1 proves that no consistent estimator of the true score covariance K_n exists without structural assumptions on the means, so sandwich and score-bootstrap inference targeting K_n^* >= K_n is asymptotically conservative rather than exact (Section 6.1, pp. 14-16).
- Confidence regions constructed naturally on the linear-system scale T_n = n^{1/2} Sigma_hat_n (beta_hat_n - beta_n) via the score bootstrap require pre-multiplying by Sigma_hat_n^{-1} or projecting onto coordinates to obtain individual slope intervals (Sections 6.3-7, pp. 19-22).
- Five minor typographical notes in arXiv:1809.10538v1 were identified and verified during full-text inspection: (1) Section 2 (page 3, line 11) prints 'A <= B is used mean that' missing 'to'; (2) Section 3 (page 8, line 15, display for gamma_n) prints '||Psi_n^{-1} Sigma_n^{-1} x_i||_2^3 = ... ||Sigma_n K_n^{-1} x_i||_2^3' with exponent -1 instead of -1/2 on Psi_n and K_n; (3) Section 6.3 (page 18, line 18) prints 'for any centrally convex set A' missing 'symmetric'; (4) Section 6.4 (page 20, line 11) prints 'represents an sample' for 'represents a sample'; and (5) Appendix A (page 29, last display equation) omits the integral sign before c(t, g_i), and Appendix A.1 (page 31, last sentence) prints 'Similar calculations holds' for 'Similar calculations hold'.
- All 33 pages of the full-text PDF of Kuchibhotla, Brown, & Buja (2018, arXiv:1809.10538v1) were read. Constructed numerical checks in research/discovery/unified-assumption-lean-moving-target-ols-and-score-bootstrap-boundaries.py verify (1) Definition 2.1 moving-target average normal equation vs. non-zero individual score expectations and positive semidefinite Loewner gap K_n^* - K_n >= 0 under i.n.i.d. nonlinearity, (2) Section 6.1 & Lemma 6.2 conservative coverage of sandwich confidence intervals via Anderson's Lemma, (3) Sections 6.2-6.4 multiplier and m-of-n resampling score bootstraps eliminating pairs-bootstrap Gram matrix singularities (0.0% vs. 3.67%) while matching K_check_n, and (4) Theorem 8.1 deterministic master inequalities across all 300 realizations satisfying D_{2n}^Sigma <= Lambda_n / 2.
Artificial Intelligence Ontology (AIO) Grounding
Mapped AIO Classes
| AIO Target Class | Mapping Relation | Mapping Rationale |
|---|---|---|
AIO:LinearRegression
Linear Regression
|
related_match | Formulates the unified assumption-lean moving-target estimation, conservative sandwich variance, score bootstrap, and deterministic master inequality theory for ordinary least squares linear regression (AIO:LinearRegression). |
Evidence Sources & Primary Literature
2 Sources| Source ID | Publication / Document | Type / Access | Reference & Checksum |
|---|---|---|---|
kuchibhotla2018-pdf
|
Arun K. Kuchibhotla, Lawrence D. Brown, Andreas Buja
|
primary paper
full_text
|
ID: https://doi.org/10.48550/arXiv.1809.10538
sha256:0306f48fd372…
|
kuchibhotla2018-abs
|
Arun Kumar Kuchibhotla, Lawrence D. Brown, Andreas Buja
|
primary paper
abstract_only
|
ID: https://doi.org/10.48550/arXiv.1809.10538
sha256:22614725f856…
|
Ontology & Grounding Topology
Open in interactive network graph →Curation Provenance
Curator: Antigravity
Method: ai_assisted
Model: Gemini 3
Created: 2026-10-17T18:15:00Z
Updated: 2026-10-17T18:15:00Z
Explore across X-Mech
Related computational & biological Mechs:
Embedding Map
Network Graph
ProteinTraitsMech (ESM-2 pLM)
DUFMech (Domain pLM)
TraitMech
CultureMech
All 14 X-Mech Repositories