Theta-IPOD Thresholding-Based Iterative Outlier Detection and Mean-Shift Penalized Regression

A thresholding-based iterative outlier detection and robust regression algorithm family ($\Theta$-IPOD, introduced by Yiyuan She and Art B. Owen in 2011) that augments the linear model $y = X\beta + \varepsilon$ with a sparse observation-level mean-shift outlier vector $\gamma \in \mathbb{R}^n$ ($y = X\beta + \gamma + \varepsilon$, $\varepsilon \sim \mathcal{N}(0, \sigma^2 I_n)$) and jointly estimates $(\hat{\beta}, \hat{\gamma})$ via the penalized criterion $\min_{\beta, \gamma} \left[\frac{1}{2}\|y - X\beta - \gamma\|_2^2 + \sum_{i=1}^n P_{\Theta}(\gamma_i; \lambda_i)\right]$ or its $H$-compressed fixed-point iteration $\gamma^{(j+1)} = \Theta(H\gamma^{(j)} + (I_n - H)y; \lambda^{(j)})$ ($H = X(X^\top X)^{-1}X^\top$, $\lambda_i^{(j)} = \lambda\sqrt{1 - h_i}\,\hat{\sigma}$). By Theorem 1, every thresholding rule $\Theta(\cdot; \lambda)$ (including soft, hard, SCAD, and hybrid hard-ridge thresholding) generates a nondecreasing penalty $P_{\Theta}(t; \lambda) = \int_0^{|t|} (\sup\{s : \Theta(s; \lambda) \le u\} - u)\,du + q(t; \lambda)$ and corresponds by Proposition 2 to a two-stage $M$-estimator with score function $\psi_{\Theta}(t; \lambda) = t - \Theta(t; \lambda)$. While convex $\ell_1$ soft-thresholding ($P_{\Theta_S} = \lambda|\gamma_1|$) reproduces Huber's $M$-estimator ($\psi_H(t; \lambda) = t - \Theta_S(t; \lambda)$) and suffers from masking and leverage vulnerability—provably failing to achieve zero false positives when $|\mathcal{O}| \ge 2$ unless an irrepresentable condition $\|(I_n - H)_{(\mathcal{O}^c)}(I_n - H)_{(\mathcal{O})}^+ \operatorname{sign}(\gamma_{\mathcal{O}})\|_\infty \le 1$ holds (Proposition 1)—nonconvex redescending thresholding rules ($\psi_{\Theta}(t; \lambda) = 0$ for $|t| \ge \lambda$, such as hard thresholding $\Theta_H$) eliminate shrinkage bias on detected outliers, achieve outlier recovery without irrepresentable conditions, and extend to high-dimensional $p > n$ settings via joint sparse $(\beta, \gamma)$ Thresholding-based Iterative Selection Procedures (TISP).