AIMech AI

Two-Stage Hard Thresholding with Voting and Plurality Rule for Invalid Instruments

Algorithms AIMech:TwoStageHardThresholdingWithVotingAndPluralityRuleForInvalidInstruments DRAFT
Aliases: TSHT, Two-Stage Hard Thresholding with Voting, Plurality-Rule Invalid Instrument Estimator, Guo-Kang-Cai-Small Two-Stage Hard Thresholding IV
A causal-effect estimation and confidence-interval construction methodology (introduced by Zijian Guo, Hyunseung Kang, T. Tony Cai, and Dylan S. Small in 2018) for linear instrumental variables models with potentially invalid instruments ($Y_i = D_i\beta^* + \mathbf{Z}_i^T\boldsymbol{\pi}^* + \mathbf{X}_i^T\boldsymbol{\phi}^* + e_i$, $D_i = \mathbf{Z}_i^T\boldsymbol{\gamma}^* + \mathbf{X}_i^T\boldsymbol{\psi}^* + \epsilon_{i2}$) that replaces the 50% majority-validity condition ($|\mathcal{V}^*| > |\mathcal{S}^*|/2$) with the strictly weaker *plurality rule* $|\mathcal{V}^*| > \max_{c \ne 0}|\{j \in \mathcal{S}^* : \pi_j^*/\gamma_j^* = c\}|$ (Assumption 1, Equation 7). Given reduced-form estimators $\widehat{\boldsymbol{\Gamma}}$ and $\widehat{\boldsymbol{\gamma}}$ (from OLS in low dimensions or Debiased Scaled Lasso in high dimensions $p \gg n$, Equations 5 and 22–26), Stage 1 estimates the set of relevant instruments $\widehat{\mathcal{S}} = \{j : |\widehat{\gamma}_j| \ge \widehat{\sigma}_2\sqrt{\|\widehat{\boldsymbol{\Omega}}_{j\cdot}\|_2^2 / n}\,\sqrt{2.01\log p_z}\}$ via first-stage hard thresholding (Equation 9); Stage 2 computes for each $j \in \widehat{\mathcal{S}}$ the ratio candidate $\widetilde{\beta}^{[j]} = \widehat{\Gamma}_j / \widehat{\gamma}_j$, noise-adjusted violation vector $\widetilde{\boldsymbol{\pi}}^{[j]} = \widehat{\boldsymbol{\Gamma}} - \widetilde{\beta}^{[j]}\widehat{\boldsymbol{\gamma}}$, and standard errors $\widehat{\mathrm{SE}}(\widetilde{\pi}_k^{[j]})$ (Equations 10–11), forms the pairwise symmetric voting matrix $\widehat{V}_{k,j} = \mathbf{1}\left(|\widetilde{\pi}_k^{[j]}| \le \widehat{\mathrm{SE}}(\widetilde{\pi}_k^{[j]})\sqrt{2.01\log|\widehat{\mathcal{S}}|}\,\text{ and }\,|\widetilde{\pi}_j^{[k]}| \le \widehat{\mathrm{SE}}(\widetilde{\pi}_j^{[k]})\sqrt{2.01\log|\widehat{\mathcal{S}}|}\right)$ (Equations 12–14), selects the valid-instrument set $\widehat{\mathcal{V}} = \arg\max_{\mathcal{V}_m}\|\widehat{\mathbf{V}}_{\cdot m}\|_0$ receiving the plurality of votes (Equations 15–16), and constructs the asymptotically normal one-step GMM point estimator $\widehat{\beta}$ and confidence interval $\mathrm{CI}_{\alpha}$ on $\widehat{\mathcal{V}}$ (Equations 17–19, Theorems 1–3).
Identity Notes & Concept Scope Resolved against all 442 pinned AIO concepts and 668 maintained AIMech concepts. Distinct from `AIMech:SomeInvalidSomeValidInstrumentalVariablesEstimatorAndMajorityRuleIdentifiability` (Kang, Zhang, Cai, & Small 2016, which uses $\ell_1$ penalization of $\boldsymbol{\alpha}$ and requires the strict 50% majority condition $|\mathcal{V}^*| > |\mathcal{S}^*|/2$ plus restricted isometry/mutual incoherence conditions for point estimation only, without providing asymptotically normal confidence intervals or handling weakly relevant instruments separated from strong ones via two-stage hard thresholding and pairwise voting), distinct from `AIMech:SelfTuningPivotalHighDimensionalInstrumentalVariablesAndSensitivityConfidenceSets` (Gautier & Rose 2011/2021, which solves a self-normalized conic program for high-dimensional endogenous regressors under user-specified instrument-invalidity sensitivity bounds $\Upsilon$ rather than identifying the valid instrument set $\mathcal{V}^*$ data-adaptively via the plurality rule and pairwise symmetric voting), and distinct from `AIMech:PenalizedFocusedGeneralizedMethodOfMomentsForHighDimensionalEndogenousRegression` (Fan & Liao 2014, which selects sparse endogenous regressors using valid over-identifying moments via folded-concave penalized Focused GMM).

Claims & Grounded Evidence

3 Claims
Mechanism #plurality-rule-identification-and-stage-1-first-stage-hard-thresholding
Under the reduced-form equations $\boldsymbol{\Gamma}^* = \beta^*\boldsymbol{\gamma}^* + \boldsymbol{\pi}^*$ with relevant instrument set $\mathcal{S}^* = \{j : \gamma_j^* \ne 0\}$ and valid subset $\mathcal{V}^* = \{j \in \mathcal{S}^* : \pi_j^* = 0\}$ (Section 2.2, p. 8, Equation 6), Assumption 1 (Equation 7) establishes that $(\beta^*, \boldsymbol{\pi}^*)$ is uniquely identified whenever $|\mathcal{V}^*| > \max_{c \ne 0}|\{j \in \mathcal{S}^* : \pi_j^* / \gamma_j^* = c\}|$ (the plurality rule), strictly relaxing the 50% majority rule $|\mathcal{V}^*| > |\mathcal{S}^*| / 2$ whenever invalid instruments do not all share the same ratio bias $\pi_j^* / \gamma_j^* = c$. Stage 1 (Section 3.1, p. 10, Equation 9) estimates $\mathcal{S}^*$ by hard thresholding $\widehat{\mathcal{S}} = \{1 \le j \le p_z : |\widehat{\gamma}_j| \ge \widehat{\sigma}_2\sqrt{\|\widehat{\boldsymbol{\Omega}}_{j\cdot}\|_2^2 / n}\,\sqrt{2.01\log p_z}\}$, screening out zero and near-zero first-stage instruments whose division $\widehat{\Gamma}_j / \widehat{\gamma}_j$ would otherwise destabilize Stage 2.
Mechanism #pairwise-symmetric-voting-matrix-and-plurality-support-recovery
For each candidate $j \in \widehat{\mathcal{S}}$, Stage 2 (Section 3.2, pp. 10–11, Equations 10–12) forms the trial causal estimator $\widetilde{\beta}^{[j]} = \widehat{\Gamma}_j / \widehat{\gamma}_j$, the invalidity residual $\widetilde{\pi}_k^{[j]} = \widehat{\Gamma}_k - \widetilde{\beta}^{[j]}\widehat{\gamma}_k$ with variance scaling $\widehat{R}_k^{[j]} = \widehat{\sigma}_1^2 + (\widetilde{\beta}^{[j]})^2\widehat{\sigma}_2^2 - 2\widetilde{\beta}^{[j]}\widehat{\sigma}_{12}$ and standard error $\widehat{\mathrm{SE}}(\widetilde{\pi}_k^{[j]}) = \sqrt{\widehat{R}_k^{[j]}\|(\widehat{\boldsymbol{\Omega}}_{k\cdot} - (\widehat{\gamma}_k / \widehat{\gamma}_j)\widehat{\boldsymbol{\Omega}}_{j\cdot})^T\mathbf{W}\mathbf{P}_{\mathbf{X}}^\perp\|_2^2 / n}$ (Equation 11), and defines the symmetric voting indicator $\widehat{V}_{k,j} = \mathbf{1}\left(|\widetilde{\pi}_k^{[j]}| \le \widehat{\mathrm{SE}}(\widetilde{\pi}_k^{[j]})\sqrt{2.01\log|\widehat{\mathcal{S}}|}\text{ and }|\widetilde{\pi}_j^{[k]}| \le \widehat{\mathrm{SE}}(\widetilde{\pi}_j^{[k]})\sqrt{2.01\log|\widehat{\mathcal{S}}|}\right)$ (Equation 14) so candidates $k$ and $j$ mutually vote for each other if and only if they yield statistically indistinguishable ratio estimates. Under Assumptions 2–5 (in low dimensions) or Assumptions 4–6 (in high dimensions $p \gg n$ via Debiased Scaled Lasso, Section 3.7, Equations 22–26), Theorem 1 (p. 20) and Theorem 3 (p. 22) prove that the plurality winner $\widehat{\mathcal{V}} = \arg\max_{\mathcal{V}_m}\|\widehat{\mathbf{V}}_{\cdot m}\|_0$ (Equation 16) satisfies $\mathcal{V}^*^{\mathrm{str}} \subseteq \widehat{\mathcal{V}} \subseteq \mathcal{V}^*$ with probability at least $1 - C p^{-c}$, and $\widehat{\mathcal{V}} = \mathcal{V}^*$ whenever all valid instruments are strong.
Scope #root-n-asymptotic-normality-and-impossibility-of-inference-without-separation
Given $\widehat{\mathcal{V}}$ from Stage 2, the one-step GMM estimator $\widehat{\beta} = \frac{\widehat{\boldsymbol{\gamma}}_{\widehat{\mathcal{V}}}^T\widehat{\mathbf{A}}\widehat{\boldsymbol{\Gamma}}_{\widehat{\mathcal{V}}}}{\widehat{\boldsymbol{\gamma}}_{\widehat{\mathcal{V}}}^T\widehat{\mathbf{A}}\widehat{\boldsymbol{\gamma}}_{\widehat{\mathcal{V}}}}$ with optimal weight matrix $\widehat{\mathbf{A}} = (\widehat{\sigma}_1^2\widehat{\boldsymbol{\Omega}}_{\widehat{\mathcal{V}}})^{-1}$ (or $\mathbf{I}_{|\widehat{\mathcal{V}}|}$ in high dimensions, Equations 17 and 27) and confidence interval $\mathrm{CI}_{\alpha} = [\widehat{\beta} - z_{1-\alpha/2}\sqrt{\widehat{V}_{\beta}/n},\, \widehat{\beta} + z_{1-\alpha/2}\sqrt{\widehat{V}_{\beta}/n}]$ (Equations 18–19) satisfy $\sqrt{n}(\widehat{\beta} - \beta^*) / \sqrt{\widehat{V}_{\beta}} \xrightarrow{d} N(0, 1)$ and $\lim_{n\to\infty}\mathbb{P}(\beta^* \in \mathrm{CI}_{\alpha}) = 1 - \alpha$ (Theorem 2, p. 21; Theorem 3, p. 22), achieving the oracle asymptotic variance $V_{\beta}^* = \sigma_1^2 / (\boldsymbol{\gamma}_{\mathcal{V}^*}^{\mathrm{str}}^T\boldsymbol{\Omega}_{\mathcal{V}^*,\mathrm{str}}^{-1}\boldsymbol{\gamma}_{\mathcal{V}^*}^{\mathrm{str}})$. Conversely, Proposition 1 (Section 4.4, p. 23, Equations 30–31) proves that if invalid instruments violate the separation condition by having local invalidity $|\pi_j^* / \gamma_j^*| \le \widetilde{\tau}_j = \sqrt{\frac{\sigma_1^2 + (\beta^* + \pi_j^*/\gamma_j^*)^2\sigma_2^2 - 2(\beta^* + \pi_j^*/\gamma_j^*)\sigma_{12}}{n}\,(\Omega_{jj} + (\gamma_j^*/\gamma_1^*)^2\Omega_{11} - 2(\gamma_j^*/\gamma_1^*)\Omega_{1j})}$, then the true parameter $(\beta^*, \boldsymbol{\pi}^*)$ is statistically indistinguishable in total variation distance from an alternative valid-instrument model with shifted causal effect $\beta^* + \pi_j^*/\gamma_j^*$ when $|\mathcal{V}^*| < p_z / 2$.

⚠️ Curated Limitations & Negative Results

Artificial Intelligence Ontology (AIO) Grounding

Mapped AIO Classes
AIO Target Class Mapping Relation Mapping Rationale
AIO:LinearRegression
Linear Regression
related_match TSHT estimates causal effects in linear structural instrumental variables regression models using two-stage hard thresholding and pairwise voting across reduced-form linear regressions.
AIO:LassoRegression
Lasso Regression
related_match High-dimensional TSHT (Section 3.7) uses Debiased Scaled Lasso for first-stage and reduced-form estimation and generalizes L1-based sisVIVE from majority rule to plurality rule.

Evidence Sources & Primary Literature

3 Sources
Source ID Publication / Document Type / Access Reference & Checksum
gkcs2018-tsht-pdf
Zijian Guo, Hyunseung Kang, T. Tony Cai, Dylan S. Small
primary paper
full_text
ID: https://doi.org/10.1111/rssb.12275
sha256:0ef075b24ff2…
gkcs2018-tsht-abs
Zijian Guo, Hyunseung Kang, T. Tony Cai, Dylan S. Small
primary paper
abstract_only
ID: https://arxiv.org/abs/1603.05224
sha256:439929441535…
gkcs2018-tsht-crossref
Zijian Guo, Hyunseung Kang, T. Tony Cai, Dylan S. Small
primary paper
abstract_only
ID: https://doi.org/10.1111/rssb.12275
sha256:992fedbb392f…

Ontology & Grounding Topology

Open in interactive network graph →
CONCEPT KIND Algorithms AIMech:TwoStageHardThresholdingWithVotingAndPluralityRuleForInvalidInstruments Two-Stage Hard... 3 claims · 3 sources AIO GROUNDING AIO:LinearRegression (2)

Curation Provenance

Curator: Antigravity
Method: ai_assisted
Model: Gemini
Created: 2026-10-19T05:00:00Z
Updated: 2026-10-19T05:00:00Z
Source File: data/curated/TwoStageHardThresholdingWithVotingAndPluralityRuleForInvalidInstruments.yaml · Schema: curated_concept.yaml

Explore across X-Mech

Related computational & biological Mechs: Embedding Map Network Graph ProteinTraitsMech (ESM-2 pLM) DUFMech (Domain pLM) TraitMech CultureMech All 14 X-Mech Repositories