Decorrelated Score Function and One-Step Estimator for Sparse High-Dimensional Inference

A general hypothesis-testing and confidence-region methodology (introduced by Yang Ning and Han Liu in 2017) for a low-dimensional or growing parameter of interest $\boldsymbol{\theta} \in \mathbb{R}^{d_0}$ in the presence of a high-dimensional sparse nuisance parameter $\boldsymbol{\gamma} \in \mathbb{R}^{d-d_0}$ ($d \gg n$) under smooth empirical risk or negative log-likelihood $\ell(\boldsymbol{\theta}, \boldsymbol{\gamma}) = n^{-1}\sum_{i=1}^n \ell_i(\boldsymbol{\theta}, \boldsymbol{\gamma})$. By projecting the target score $\nabla_{\boldsymbol{\theta}}\ell(\boldsymbol{\theta}, \boldsymbol{\gamma})$ onto the nuisance tangent space via $\mathbf{w}^* = (\mathbf{I}_{\boldsymbol{\gamma}\boldsymbol{\gamma}}^*)^{-1}\mathbf{I}_{\boldsymbol{\gamma}\boldsymbol{\theta}}^*$ where $\mathbf{I}^* = \mathbb{E}_{\boldsymbol{\beta}^*}(\nabla^2\ell(\boldsymbol{\beta}^*))$, the method defines the population *decorrelated score function* $S(\boldsymbol{\theta}, \boldsymbol{\gamma}) = \nabla_{\boldsymbol{\theta}}\ell(\boldsymbol{\theta}, \boldsymbol{\gamma}) - \mathbf{w}^{*T}\nabla_{\boldsymbol{\gamma}}\ell(\boldsymbol{\theta}, \boldsymbol{\gamma})$ satisfying $\mathbb{E}_{\boldsymbol{\beta}^*}[\nabla_{\boldsymbol{\theta}}\ell(\boldsymbol{\theta}^*, \boldsymbol{\gamma}^*)S^T(\boldsymbol{\theta}^*, \boldsymbol{\gamma}^*)] = \mathbf{0}$ (Equation 2.4). Given an initial penalized M-estimator $\widehat{\boldsymbol{\beta}} = (\widehat{\boldsymbol{\theta}}, \widehat{\boldsymbol{\gamma}})$ (Equation 2.5) and a Dantzig-selector or penalized quadratic projection estimator $\widehat{\mathbf{w}}$ (Equations 2.6–2.7), the plug-in decorrelated score $\widehat{S}(\boldsymbol{\theta}, \widehat{\boldsymbol{\gamma}}) = \nabla_{\boldsymbol{\theta}}\ell(\boldsymbol{\theta}, \widehat{\boldsymbol{\gamma}}) - \widehat{\mathbf{w}}^T\nabla_{\boldsymbol{\gamma}}\ell(\boldsymbol{\theta}, \widehat{\boldsymbol{\gamma}})$ converts first-order nuisance estimation bias into the second-order Taylor product $(\widehat{\mathbf{w}} - \mathbf{w}^*)^T\nabla^2_{\boldsymbol{\gamma}\boldsymbol{\gamma}}\ell(\boldsymbol{\theta}, \widetilde{\boldsymbol{\gamma}})(\widehat{\boldsymbol{\gamma}} - \boldsymbol{\gamma}^*)$ without inverting the full $d \times d$ Hessian. This yields the asymptotically normal score statistic $\widehat{U}_n = \sqrt{n}\widehat{S}(\mathbf{0}, \widehat{\boldsymbol{\gamma}})\widehat{I}_{\theta|\boldsymbol{\gamma}}^{-1/2} \rightsquigarrow N(0, 1)$ where $\widehat{I}_{\theta|\boldsymbol{\gamma}} = \nabla^2_{\theta\theta}\ell(\widehat{\boldsymbol{\beta}}) - \widehat{\mathbf{w}}^T\nabla^2_{\boldsymbol{\gamma}\theta}\ell(\widehat{\boldsymbol{\beta}})$ (Algorithm 1, Theorem 3.7), the semiparametrically efficient *one-step estimator* $\widetilde{\theta} = \widehat{\theta} - \widehat{S}(\widehat{\boldsymbol{\beta}})/\widehat{I}_{\theta|\boldsymbol{\gamma}}$ (Equation 2.9, Corollary 3.10), the Gaussian multiplier bootstrap supremum test $\widehat{T}_n = \|\sqrt{n}\widehat{S}(\mathbf{0}, \widehat{\boldsymbol{\gamma}})\|_\infty$ for high-dimensional nulls $H_0: \boldsymbol{\theta}^* = \mathbf{0} \in \mathbb{R}^{d_0}$ (Section 6.1, Theorem 6.1), and sandwich-variance-adjusted tests under model misspecification and general non-likelihood losses (Sections 6.2–6.3, Theorems 6.4 and 6.8).