Deterministic Banach contraction and Newton-Kantorovich inequalities for smooth M-estimators
Mathematical Methods AIMech:DeterministicBanachContractionAndNewtonKantorovichInequalitiesForSmoothMEstimators DRAFT
Aliases: Banach Contraction Inversion and Semilocal Newton-Kantorovich Bahadur Expansions for M-Estimators, Generalized Self-Concordant Hessian-Ratio C(u, w) Deterministic M-Estimation Inequalities, Deterministic Leave-k-Out Cross-Validation and Cox Partial Likelihood Bahadur Bounds
Deterministic Banach contraction and Newton-Kantorovich inequalities for smooth M-estimators are a finite-sample, assumption-free analytical framework introduced by Kuchibhotla (2018, arXiv:1809.05172v1) that decouples the existence, localization, and first-order Bahadur influence-function expansion of zeros of smooth estimating equations f(theta) = 0 from any probabilistic or i.i.d. data-generating assumptions. Using the Banach fixed-point theorem applied to phi(theta) := theta - A^{-1} f(theta) on B(theta_0, r), Theorem 2.1 proves that whenever sup_{theta in B(theta_0, r)} ||A^{-1}(A - nabla f(theta))||_{op} <= varepsilon < 1 and ||A^{-1} f(theta_0)||_2 <= r(1 - varepsilon) for an arbitrary reference point theta_0 (not necessarily satisfying E[f(theta_0)] = 0), there exists a unique root theta^* in B(theta_0, r) satisfying the two-sided norm sandwiching (1 + varepsilon)^{-1} ||A^{-1} f(theta_0)||_2 <= ||theta^* - theta_0||_2 <= (1 - varepsilon)^{-1} ||A^{-1} f(theta_0)||_2; under local Holder Jacobian continuity ||[nabla f(theta_0)]^{-1}(nabla f(theta_0) - nabla f(theta))||_{op} <= L ||theta - theta_0||_2^alpha, Theorem 2.2 establishes the semilocal Newton-Kantorovich Bahadur expansion ||theta^* - theta_0 + [nabla f(theta_0)]^{-1} f(theta_0)||_2 <= (1.5)^{1+alpha} L ||[nabla f(theta_0)]^{-1} f(theta_0)||_2^{1+alpha}. For smooth convex losses L(theta; w), Section 3 introduces the scale-invariant directional Hessian-ratio modulus C(u, w) := sup_{||theta_1 - theta_2||_2 <= u} sup_{||e||_2 = 1} (e^T nabla_2 L(theta_1, w) e) / (e^T nabla_2 L(theta_2, w) e) >= 1 (forming a convex cone C_T under nonnegative combinations, Proposition 3.1) and proves in Theorem 3.1 that whenever max_{1<=i<=n} C(delta_n(theta_0), W_i) <= 4/3 with delta_n(theta_0) := 1.5 ||[Q_hat_n(theta_0)]^{-1} Z_hat_n(theta_0)||_2, a root theta_hat_n exists with (1/2) delta_n(theta_0) <= ||theta_hat_n - theta_0||_2 <= delta_n(theta_0) and ||theta_hat_n - theta_0 + [Q_hat_n(theta_0)]^{-1} Z_hat_n(theta_0)||_2 <= max_{1<=i<=n} {C(delta_n(theta_0), W_i) - 1} delta_n(theta_0), extending directly to generalized linear models (Corollary 3.1), Cox proportional hazards partial likelihood (Theorem 4.1), non-convex least squares (Corollary 5.1), equality-constrained KKT systems (Corollary 6.1), and sample-computable leave-k-out cross-validation, marginal screening, and uniform-in-submodel post-selection inference (Corollaries 7.1-7.4).
Identity Notes & Concept Scope
Evaluated against all 442 pinned AIO records and 614 maintained curated concepts in AIMech. Distinct from AIMech:UniformInSubmodelDeterministicLinearRegressionAndFunctionalDependenceBounds (Kuchibhotla et al. 2021, arXiv:1802.05801v3) and AIMech:UnifiedAssumptionLeanMovingTargetLinearRegressionAndScoreBootstrap (Kuchibhotla, Brown, & Buja 2018, arXiv:1809.10538v1), which analyze closed-form linear least-squares estimators beta_M(Sigma, Gamma) = [Sigma(M)]^{-1} Gamma(M) whose normal equations are already linear in beta (where C(u, y) = 1 identically), whereas the present concept establishes general Banach contraction fixed-point inversion (Theorem 2.1), semilocal Newton-Kantorovich Holder expansions (Theorem 2.2), generalized self-concordant directional Hessian-ratio bounds C(u, w) <= 4/3 (Theorem 3.1, Proposition 3.1), Cox proportional hazards non-sample-average partial likelihood inequalities (Theorem 4.1, Lemma C.1), non-convex nonlinear least squares (Corollary 5.1), equality-constrained KKT saddle systems (Corollary 6.1), and sample-computable leave-|I|-out cross-validation expansions around theta_0 = theta_hat_n (Corollaries 7.1-7.2). Distinct from AIMech:LocalRademacherComplexityAndBousquetInequality, AIMech:TalagrandConvexDistanceAndSupremaConcentrationInequalities, and AIMech:GeneralizedBernsteinOrliczSubWeibullConcentrationAndHighDimensionalLasso, which provide probabilistic concentration inequalities rather than deterministic root-existence and Bahadur expansion theorems.
Claims & Grounded Evidence
4 Claims
Mechanism
#banach-contraction-inversion-and-newton-kantorovich-holder-bahadur-expansion
Section 2 and Appendix A (pp. 5-8 and 35-36, Theorems 2.1 and 2.2, Remark 2.1, Equations 3-7 and 30-31) prove via the Banach contraction mapping theorem applied to phi(theta) := theta - A^{-1} f(theta) on B(theta_0, r) that whenever sup_{theta in B(theta_0, r)} ||A^{-1}(A - nabla f(theta))||_{op} <= varepsilon < 1 (Equation 3) and ||A^{-1} f(theta_0)||_2 <= r(1 - varepsilon) (Equation 4), there exists a unique root theta^* in B(theta_0, r) of f(theta^*) = 0 satisfying the two-sided sandwiching (1 + varepsilon)^{-1} ||A^{-1} f(theta_0)||_2 <= ||theta^* - theta_0||_2 <= (1 - varepsilon)^{-1} ||A^{-1} f(theta_0)||_2 (Theorem 2.1); under local Holder Jacobian continuity ||[nabla f(theta_0)]^{-1}(nabla f(theta_0) - nabla f(theta))||_{op} <= L ||theta - theta_0||_2^alpha for (3L)^{1/alpha} ||theta - theta_0||_2 <= 1 (Equation 5) and ||[nabla f(theta_0)]^{-1} f(theta_0)||_2 <= 2 / (3 (3L)^{1/alpha}) (Equation 6), Theorem 2.2 proves unique root existence in B(theta_0, 1.5 ||[nabla f(theta_0)]^{-1} f(theta_0)||_2) and the first-step Newton / Bahadur expansion ||theta^* - theta_0 + [nabla f(theta_0)]^{-1} f(theta_0)||_2 <= (1.5)^{1+alpha} L ||[nabla f(theta_0)]^{-1} f(theta_0)||_2^{1+alpha} (Equation 7), without requiring E[f(theta_0)] = 0 or convexity.
Mechanism
#generalized-self-concordant-hessian-ratio-modulus-and-convex-cone-property
Section 3 and Appendix B (pp. 8-17 and 36-39, Theorem 3.1, Corollary 3.1, Proposition 3.1, Remarks 3.1-3.3, Examples 3.1-3.3, Equations 8-13 and 32-36) define the directional Hessian-ratio function C(u, w) := sup_{||theta_1 - theta_2||_2 <= u} sup_{||e||_2 = 1} (e^T nabla_2 L(theta_1, w) e) / (e^T nabla_2 L(theta_2, w) e) >= 1 and prove in Theorem 3.1 that on the event E_n := {max_{1<=i<=n} C(delta_n(theta_0), W_i) <= 4/3} with delta_n(theta_0) := 1.5 ||[Q_hat_n(theta_0)]^{-1} Z_hat_n(theta_0)||_2, there exists theta_hat_n in R^q solving Z_hat_n(theta_hat_n) = 0 with (1/2) delta_n(theta_0) <= ||theta_hat_n - theta_0||_2 <= delta_n(theta_0) (Equation 9) and ||theta_hat_n - theta_0 + [Q_hat_n(theta_0)]^{-1} Z_hat_n(theta_0)||_2 <= max_{1<=i<=n} {C(delta_n(theta_0), W_i) - 1} delta_n(theta_0) (Equation 10), while Proposition 3.1 proves via the mediant inequality min{a/b, c/d} <= (a + c)/(b + d) <= max{a/b, c/d} (Equation 35) that the class C_T of losses satisfying C(u, w) <= T(u, w) forms a convex cone closed under nonnegative linear combinations and convex penalties, and Corollary 3.1 / Example 3.3 specialize to generalized linear models with C(u, y) = 1 (linear), e^u (Poisson), and <= e^{3u} (logistic and negative binomial) without artificial lower-curvature assumptions.
Mechanism
#cox-partial-likelihood-nonlinear-least-squares-and-kkt-constrained-extensions
Sections 4-6 and Appendices C-E (pp. 17-23 and 39-46, Theorem 4.1, Corollaries 5.1 and 6.1, Lemma C.1, Equations 14-22 and 37-45) extend the deterministic fixed-point and Bahadur inequalities beyond sample-average convex losses: for the generalized weighted Cox proportional hazards partial log-likelihood L_hat_n(beta) (Equation 14), Lemma C.1 bounds the second-derivative ratio of K(t) = log(sum_{i=1}^n w_i e^{a_i t}) by |K''(t)/K''(0) - 1| <= 4 mu_n |t| exp(4 mu_n |t|) where mu_n := max_{1<=i<=n} |a_i|, yielding Theorem 4.1 that whenever sup_{0<=s<infty} mu_n(s) delta_n(beta_0) <= 1/16, beta_hat_n exists with (1/2) delta_n(beta_0) <= ||beta_hat_n - beta_0||_2 <= delta_n(beta_0) and ||beta_hat_n - beta_0 + [Q_hat_n(beta_0)]^{-1} Z_hat_n(beta_0)||_2 <= 8 e^{1/4} delta_n^2(beta_0) sup_{0<=s<infty} mu_n(s) (Equation 15); Corollary 5.1 establishes deterministic root existence and expansion with polynomial-Holder modulus omega(r) = L_2(theta_0) r^2 + L_{1+alpha}(theta_0) r^{1+alpha} + L_1(theta_0) r + L_alpha(theta_0) r^alpha for non-convex nonlinear least squares F_n(theta) = n^{-1} sum_{i=1}^n (Y_i - g(theta^T X_i))^2 (Equation 16); and Corollary 6.1 establishes deterministic existence and expansion for equality-constrained optimization min_beta F_n(beta) subject to A beta = b via the KKT block Jacobian J_n(beta_0) = [nabla_2 F_n(beta_0)](I - [nabla_2 F_n(beta_0)]^{-1} A^T (A [nabla_2 F_n(beta_0)]^{-1} A^T)^{-1} A)^{-1} (Equations 17 and 20-21).
Scope
#leave-k-out-cross-validation-marginal-screening-and-uniform-submodel-expansions
Section 7 and Appendix F (pp. 23-30 and 47-49, Corollaries 7.1-7.4, Remarks 7.1-7.4, Equations 23-29 and 46-49) exploit the freedom to choose a data-dependent reference point theta_0 = theta_hat_n in Theorem 3.1 to prove purely sample-computable deterministic bounds for leave-|I|-out estimators theta_hat_{-I} = argmin_theta sum_{i notin I} L(theta, W_i): under max_{i notin I} C(1.5 delta_{I,n}, W_i) <= 4/3 where delta_{I,n} := ||[(n - |I|) Q_hat_{-I}(theta_hat_n)]^{-1} sum_{i in I} nabla L(theta_hat_n, W_i)||_2, Corollary 7.2 proves (3/4) delta_{I,n} <= ||theta_hat_{-I} - theta_hat_n||_2 <= (3/2) delta_{I,n} and ||theta_hat_{-I} - theta_hat_n - n^{-1} Q_hat_n^{-1} sum_{i in I} nabla L(theta_hat_n, W_i)||_2 <= (3/2) delta_{I,n} [max_{i notin I} C(1.5 delta_{I,n}, W_i) - 1 + n^{-1} ||Q_hat_n^{-1} sum_{i in I} nabla_2 L(theta_hat_n, W_i)||_{op}] (Equations 25-26), yielding O_p(|I|^2 n^{-2}) expansion error when C(., w) is differentiable at 0; Corollaries 7.3 and 7.4 derive uniform deterministic linear representations for marginal screening estimators {beta_hat_{n,j}}_{1<=j<=p} (Equation 28) and submodel M-estimators {beta_hat_{n,M}}_{M in M} for post-selection inference (Equation 29).
⚠️ Curated Limitations & Negative Results
- Smoothness / Differentiability Requirement on the Estimating Equation: Theorems 2.1, 2.2, 3.1, and 4.1 require the estimating equation f(theta) = Z_hat_n(theta) to be everywhere differentiable (or the loss L(theta; w) to be twice differentiable with non-singular empirical Hessian Q_hat_n(theta_0)); non-smooth M-estimators such as median/quantile regression or hinge-loss SVM require smooth convolution/mollifier approximation as discussed in Section 8.1.
- Local Contraction Radius and Empirical Hessian Invertibility Condition: The deterministic guarantees in Theorems 2.2 and 3.1 hold conditional on the empirical Hessian Q_hat_n(theta_0) (or block Jacobian J_n(beta_0)) being invertible and the scaled Newton step delta_n(theta_0) := 1.5 ||[Q_hat_n(theta_0)]^{-1} Z_hat_n(theta_0)||_2 satisfying the local Hessian-ratio bound max_{1<=i<=n} C(delta_n(theta_0), W_i) <= 4/3 (or sup_s mu_n(s) delta_n(beta_0) <= 1/16 for the Cox partial likelihood in Theorem 4.1).
- Primary-Source Typographical Notes in arXiv:1809.05172v1: Verified five self-correcting typographical slips in the primary manuscript: (1) Example 3.3.1 (p. 15, first display) prints 'beta_hat_0 - beta_0' instead of 'beta_hat_n - beta_0'; (2) Corollary 6.1 (p. 22, Equation 21) prints a minus sign '-[J_n(beta_0)]^{-1}(nabla F_n(beta_0) + A^T nu_0)' inside the norm instead of '+[J_n(beta_0)]^{-1}(nabla F_n(beta_0) + A^T nu_0)' (cf. Theorem 2.2 Equation 7 and Appendix E.1 p. 46); (3) Appendix A.1 (p. 35, line above Equation 30) prints 'nabla f(theta_bar)(theta - theta_0)' for 'nabla f(theta_bar)(theta^* - theta_0)', and Appendix B.1 (p. 37) prints 'nabla L' for 'nabla_2 L' in Equation (32) and 'nabla f(theta_0)' for 'nabla f(theta)' inside the supremum in Equation (33); (4) Appendix C.2 (p. 43, display 4) prints upper bound '2 delta_n(beta_0)' instead of 'delta_n(beta_0)' (cf. Theorem 4.1 on p. 19), and p. 44 (top display) prints '[J_hat_n(beta_0)]^{-1}' and 'delta^2(beta_0)' for '[Q_hat_n(beta_0)]^{-1}' and 'delta_n^2(beta_0)'; and (5) Appendix F (p. 48, Proof of Corollary 7.2) prints 'W_I' inside 'sum_{i in I} nabla L(theta_hat_n, W_I)' instead of 'W_i', and Equation (26) (p. 25) and Equation (47) (p. 48) print 'i != I' instead of 'i notin I'.
- All 49 pages of the full-text PDF of Kuchibhotla (2018, arXiv:1809.05172v1) were read. Constructed numerical checks in research/discovery/deterministic-inequalities-smooth-m-estimators-boundaries.py verify (1) Theorems 2.1 and 2.2 Banach contraction root sandwiching and Newton-Kantorovich Holder Bahadur expansion on a 2D nonlinear system, (2) Theorem 3.1, Corollary 3.1, and Proposition 3.1 directional Hessian-ratio bounds and convex-cone mediant inequality for logistic-ridge M-estimation, (3) Lemma C.1 and Theorem 4.1 Cox proportional hazards partial likelihood bounds, and (4) Corollaries 6.1 and 7.2 equality-constrained KKT block Jacobian inversion and leave-one-out deterministic expansion.
Artificial Intelligence Ontology (AIO) Grounding
Mapped AIO Classes
| AIO Target Class | Mapping Relation | Mapping Rationale |
|---|---|---|
AIO:LinearRegression
Linear Regression
|
related_match | AIO:LinearRegression represents linear regression broadly, whereas AIMech:DeterministicBanachContractionAndNewtonKantorovichInequalitiesForSmoothMEstimators formulates general Banach contraction fixed-point inversion and Newton-Kantorovich / generalized self-concordant Bahadur expansion inequalities encompassing ordinary and nonlinear least squares as special cases (Example 3.3.1, Corollary 5.1). |
AIO:LogisticRegression
Logistic Regression
|
related_match | AIO:LogisticRegression represents logistic regression broadly, whereas AIMech:DeterministicBanachContractionAndNewtonKantorovichInequalitiesForSmoothMEstimators establishes deterministic existence and Bahadur expansion bounds for logistic regression via the directional Hessian-ratio bound C(u, y) <= e^{3u} without lower-curvature assumptions (Example 3.3.3, Corollary 3.1). |
Evidence Sources & Primary Literature
2 Sources| Source ID | Publication / Document | Type / Access | Reference & Checksum |
|---|---|---|---|
kuchibhotla2018-det-m-est-pdf
|
Arun Kumar Kuchibhotla
|
primary paper
full_text
|
ID: https://doi.org/10.48550/arXiv.1809.05172
sha256:8a99157a7563…
|
kuchibhotla2018-det-m-est-abs
|
Arun Kumar Kuchibhotla
|
primary paper
abstract_only
|
ID: https://doi.org/10.48550/arXiv.1809.05172
sha256:3bd36a9b921c…
|
Ontology & Grounding Topology
Open in interactive network graph →Curation Provenance
Curator: Antigravity
Method: ai_assisted
Model: Gemini 3
Created: 2026-10-18T19:30:00Z
Updated: 2026-10-18T19:30:00Z
Explore across X-Mech
Related computational & biological Mechs:
Embedding Map
Network Graph
ProteinTraitsMech (ESM-2 pLM)
DUFMech (Domain pLM)
TraitMech
CultureMech
All 14 X-Mech Repositories