AIMech AI

Simes closed-testing concentration-set shortcut and double-asymptotic false discovery proportion consistency

Mathematical Methods AIMech:SimesClosedTestingConcentrationShortcutAndDoubleAsymptoticFDPConsistency DRAFT
Aliases: Goeman-Meijer-Krebs-Solari exact Simes closed-testing FDP shortcut, Simes closed-testing concentration set Z_alpha, Double-limit consistency of Simes closed-testing FDP confidence bounds, Simes-detectable non-consonant closed testing
Simes closed-testing concentration-set shortcut and double-asymptotic false discovery proportion (FDP) consistency is the multiple-testing methodology introduced by Goeman, Meijer, Krebs, and Solari (2019) for closed testing (Marcus, Peritz, & Gabriel 1976; Goeman & Solari 2011) with Simes local tests I in U_alpha iff min_{1 <= i <= |I|} |I| * p_{(i:I)} / i <= alpha across m hypotheses H_1, ..., H_m with true null index set T subseteq {1, ..., m}. Letting K_i = {r_{m-i+1}, ..., r_m} denote the index set of the i largest p-values and defining Hommel's worst-case non-rejected intersection size h_alpha = max{0 <= i <= m : K_i not in U_alpha} (which yields the simultaneous (1 - alpha)-upper confidence bound pi_hat_alpha = h_alpha / m on |T| / m), the method characterizes closed-testing rejections X_alpha = {I : forall J supseteq I, J in U_alpha} via I in X_alpha iff exists 1 <= i <= |I| such that h_alpha * p_{(i:I)} <= i * alpha (Lemma 2), and establishes the exact all-subset shortcut (Theorem 1) for the simultaneous (1 - alpha)-lower bound d_alpha(S) = |S| - t_alpha(S) on true discoveries |S \ T| and upper bound q_alpha(S) = t_alpha(S) / |S| on the FDP pi(S) = |T cap S| / |S|: d_alpha(S) = max_{1 <= u <= |S|} (1 - u + |{i in S : h_alpha * p_i <= u * alpha}|), where the maximization over u restricts to 1 <= u <= z_alpha - m + h_alpha + 1 with z_alpha = min{m - h_alpha <= i <= m : h_alpha * p_{(i)} <= (i - m + h_alpha + 1) * alpha} for h_alpha < m and 0 for h_alpha = m (Lemma 3), enabling O(|S|) per-subset evaluation after a one-time O(m log m) computation of h_alpha. The concentration set Z_alpha = L_{z_alpha} of the z_alpha smallest p-values is the unique minimal set satisfying d_alpha(S) = d_alpha(S cap Z_alpha) for all S subseteq {1, ..., m}, with d_alpha(Z_alpha) = m - h_alpha and |S cap R_alpha| <= d_alpha(S) <= |S cap Z_alpha| <= |S cap B_alpha| between Hommel's elementary FWER set R_alpha and the Benjamini-Hochberg (BH) set B_alpha = L_{b_alpha} (Lemma 4), while alpha * q_alpha(B_q) <= pi_hat_alpha * q holds simultaneously for all q in [0, 1] (Lemma 5). Under Efron's mixture model P_n(x) = gamma * x + (1 - gamma) * P_{1,n}(x), Non-consonant Simes rejections yield non-vanishing average discovery power whenever P_n is Simes-detectable (bar_pi_alpha = inf_{0 <= x < 1} (1 - P_n(x * alpha)) / (1 - x) < 1, Lemma 6 and Theorem 2) even though |R_alpha| / m ->^P 0, and for any subset sequence S_m with plim_{m -> infty} |S_m| / m = c > 0 and consistent alternative tests lim_{n -> infty} P^S_{1,n}(x) = 1, q_{alpha,m,n}(S_m) is a doubly asymptotically consistent estimator of the true subset FDP gamma_S = plim_{m -> infty} pi(S_m) as (m, n) -> infty at arbitrary relative rates for any fixed alpha in (0, 1) (Theorem 3).
Identity Notes & Concept Scope Distinct from AIMech:ReferenceFamilyJERPostHocConfidenceBounds (Blanchard, Neuvial, & Roquain 2020, The Annals of Statistics 48(3):1281-1303, arXiv:1703.02307), which formulates general reference-family joint error rate (JER) interpolation V_bar_{R_ref}(R), single-step/step-down lambda-calibration under known and unknown dependence, and balanced-template Higher Criticism optimality; Goeman, Meijer, Krebs, & Solari (2019, Biometrika 106(4):841-856, arXiv:1611.06739v2, DOI:10.1093/biomet/asz041) specifically analyze closed testing with Simes local tests via Hommel's global worst-case size h_alpha, derive the exact O(m log m) + O(|S|) counting-sort shortcut d_alpha(S) = max_{1 <= u <= |S|} (1 - u + |{i in S : h_alpha * p_i <= u * alpha}|) and the unique minimal concentration set Z_alpha = L_{z_alpha} sandwiching discoveries between Hommel's FWER set R_alpha and Benjamini-Hochberg's FDR set B_alpha, prove the simultaneous adaptive BH tail bound alpha * q_alpha(B_q) <= pi_hat_alpha * q, prove that non-consonant rejections drive non-vanishing large-m discovery power under Simes-detectability while consonant FWER rejections vanish (|R_alpha|/m ->^P 0), and prove double-limit consistency plim_{(m,n)->infty} q_{alpha,m,n}(S_m) = gamma_S at arbitrary relative rates of (m, n) via Hildebrandt's iterated-limit theorem. Also distinct from AIMech:ForestStructuredDKWPostHocConfidenceBounds (Durand, Blanchard, Neuvial, & Roquain 2020) and AIMech:HadamardDifferentiableThresholdFDPAsymptoticsAndFixedPointDuality (Neuvial 2008).

Claims & Grounded Evidence

4 Claims
Mechanism #hommel-h-characterization-and-exact-linear-time-all-subset-shortcut
In closed testing with Simes local tests I in U_alpha iff min_{1 <= i <= |I|} |I| * p_{(i:I)} / i <= alpha, Hommel's worst-case non-rejected intersection size h_alpha = max{0 <= i <= m : K_i not in U_alpha} (computed in O(m log m) time via bisection on the i largest p-values K_i = {r_{m-i+1}, ..., r_m}) satisfies t_alpha({1, ..., m}) = h_alpha, yielding the (1 - alpha)-upper confidence bound pi_hat_alpha = h_alpha / m on the true null proportion |T| / m (Lemma 1, Equation 9), and characterizes closed-testing rejections via I in X_alpha iff exists 1 <= i <= |I| such that h_alpha * p_{(i:I)} <= i * alpha (Lemma 2). Consequently, for every nonempty subset S subseteq {1, ..., m}, the simultaneous (1 - alpha)-lower confidence bound on true discoveries d_alpha(S) = |S| - t_alpha(S) is given exactly by d_alpha(S) = max_{1 <= u <= |S|} (1 - u + |{i in S : h_alpha * p_i <= u * alpha}|) (Theorem 1, Equation 10), where the maximizing u is bounded above by z_alpha - m + h_alpha + 1 with z_alpha = min{m - h_alpha <= i <= m : h_alpha * p_{(i)} <= (i - m + h_alpha + 1) * alpha} for h_alpha < m and 0 for h_alpha = m (Lemma 3), allowing exact evaluation in O(|S|) time via counting sort on c_i = ceil(h_alpha * p_i / alpha) and strictly improving upon the approximate O(|S|^2) shortcut of Goeman & Solari (2011) (e.g., on the Figure 1 example with m = 4 and alpha/2 < p_1 <= p_2 <= p_3 <= 2*alpha/3 < alpha < p_4, where R_alpha = empty, t_alpha({1,2,3}) = 1 < |{1,2,3} \ R_alpha| = 3, and t_alpha({1,2}) = 1 whereas the 2011 shortcut fails on L_2 = {1,2} with bound 2).
Mechanism #concentration-set-minimality-and-benjamini-hochberg-sandwich-bounds
Defining the concentration set Z_alpha = L_{z_alpha} of the z_alpha smallest p-values and the Benjamini-Hochberg rejection set B_alpha = L_{b_alpha} with b_alpha = max{1 <= i <= m : m * p_{(i)} <= i * alpha} (Equation 11), Lemma 4 proves that: (1) d_alpha(S) = d_alpha(S cap Z_alpha) for every S subseteq {1, ..., m}, so no hypothesis outside Z_alpha contributes to any non-trivial discovery bound; (2) Z_alpha is the unique minimal set with property (1) (if d_alpha(S) = d_alpha(S cap Y) for all S, then Z_alpha subseteq Y); (3) d_alpha(Z_alpha) = m - h_alpha; and (4) m - h_alpha <= z_alpha <= b_alpha (Z_alpha subseteq B_alpha), yielding the universal discovery sandwich |S cap R_alpha| <= d_alpha(S) <= |S cap Z_alpha| <= |S cap B_alpha| and the non-trivial BH-set FDP bound q_alpha(B_alpha) = (b_alpha - m + h_alpha) / b_alpha < 1 whenever b_alpha > 0. Moreover, for any q in [0, 1], Lemma 5 proves alpha * q_alpha(B_q) <= pi_hat_alpha * q (with equality iff pi_hat_alpha * q = 0, and more strongly q_alpha(B_q) <= pi_hat_alpha * q / alpha - 1 / b_q when b_q > 0 and pi_hat_alpha * q > 0), strictly improving upon the Lehmann-Romano (2005) Markov bound q / alpha simultaneously over all q and bounding the simultaneously median-unbiased estimator q_hat(B_q) = q_{1/2}(B_q) <= 2 * q * pi_hat_{1/2} (Equation 12).
Mechanism #simes-detectability-limit-and-non-vanishing-non-consonant-power
Under Efron's mixture model P_n(x) = gamma * x + (1 - gamma) * P_{1,n}(x) (Equation 13) with independent (or Glivenko-Cantelli weakly dependent) p-values, as m -> infty at fixed sample size n, Lemma 6 and Lemma 7 prove that pi_hat_alpha = h_alpha / m ->^P bar_pi_alpha = inf_{0 <= x < 1} (1 - P(x * alpha)) / (1 - x) > 0 when P(alpha) < 1 (and 0 otherwise). While Hommel's elementary FWER rejection set R_alpha = {1 <= i <= m : h_alpha * p_i <= alpha} has vanishing discovery fraction |R_alpha| / m ->^P 0 whenever P(alpha) < 1 and P(0) = 0 (Equation 14), Theorem 2 proves that for any q in [0, 1] such that P is Simes-detectable at q * alpha (meaning bar_pi_{q*alpha} < 1, i.e., P(x * q * alpha) > x for some x in [0, 1) or P(q * alpha) = 1), the data-dependent set J_m = B_{tilde_alpha_m, m} with tilde_alpha_m = q * alpha * bar_pi_alpha / pi_hat_{alpha,m} satisfies q_alpha(J_m) <= bar_pi_alpha * q for all m and lim_{m -> infty} P(|J_m| / m >= y) = 1 for some y > 0, establishing non-vanishing average discovery power driven asymptotically by non-consonant rejections.
Mechanism #double-asymptotic-fdp-consistency-at-arbitrary-relative-rates
For any infinite target subset S subseteq N whose finite-m truncation S_m = S cap {1, ..., m} has non-vanishing relative size plim_{m -> infty} |S_m| / m = c > 0 (Equation 17) and whose conditional p-value distribution P^S_n(x) = gamma_S * x + (1 - gamma_S) * P^S_{1,n}(x) (Equation 15) has a consistent alternative lim_{n -> infty} P^S_{1,n}(x) = 1 for all x in (0, 1] that is weakly increasing in n (Equation 16), Theorem 3 and Hildebrandt's iterated-limit theorem (Lemma 8) prove that for every fixed confidence level alpha in (0, 1), plim_{(m,n) -> infty} q_{alpha,m,n}(S_m) = gamma_S = plim_{m -> infty} pi(S_m) at arbitrary relative growth rates of m and n, whereas the Benjamini-Hochberg rejection set at fixed alpha satisfies plim_{(m,n) -> infty} |B_{alpha,m,n}| / m = (1 - gamma) / (1 - alpha * gamma) > 1 - gamma whenever gamma > 0.

⚠️ Curated Limitations & Negative Results

Artificial Intelligence Ontology (AIO) Grounding

Mapped AIO Classes
AIO Target Class Mapping Relation Mapping Rationale
AIO:DataDredgingBias
Data Dredging Bias
related_match Simultaneous closed-testing FDP confidence bounds across all 2^m subsets protect against post hoc data dredging and arbitrary subset revision (AIO:DataDredgingBias).

Evidence Sources & Primary Literature

3 Sources
Source ID Publication / Document Type / Access Reference & Checksum
gmks2019-pdf
Jelle J. Goeman, Rosa J. Meijer, Thijmen J. P. Krebs, Aldo Solari
primary paper
full_text
ID: https://doi.org/10.1093/biomet/asz041
sha256:a3f4992cffde…
gmks2019-crossref
Jelle J. Goeman, Rosa J. Meijer, Thijmen J. P. Krebs, Aldo Solari
primary paper
abstract_only
ID: https://doi.org/10.1093/biomet/asz041
sha256:ab62d4953d0e…
gmks2019-abs
Jelle J. Goeman, Rosa J. Meijer, Thijmen J. P. Krebs, Aldo Solari
primary paper
abstract_only
ID: https://doi.org/10.48550/arXiv.1611.06739
sha256:c162b611101e…

Ontology & Grounding Topology

Open in interactive network graph →
CONCEPT KIND Mathematical Methods AIMech:SimesClosedTestingConcentrationShortcutAndDoubleAsymptoticFDPConsistency Simes closed-testing... 4 claims · 3 sources AIO GROUNDING AIO:DataDredgingBias (1)

Curation Provenance

Curator: Antigravity
Method: ai_assisted
Model: Gemini 3
Created: 2026-10-17T17:15:00Z
Updated: 2026-10-17T17:15:00Z
Source File: data/curated/SimesClosedTestingConcentrationShortcutAndDoubleAsymptoticFDPConsistency.yaml · Schema: curated_concept.yaml

Explore across X-Mech

Related computational & biological Mechs: Embedding Map Network Graph ProteinTraitsMech (ESM-2 pLM) DUFMech (Domain pLM) TraitMech CultureMech All 14 X-Mech Repositories