Locally Simultaneous Inference

TL;DR

Locally simultaneous inference offers a less conservative, data-driven approach to selective inference, providing valid answers to plausible questions while maintaining statistical power and applicability in nonparametric settings.

Contribution

It introduces a novel locally simultaneous inference method that refines existing simultaneous inference, reducing conservativeness and improving applicability and stability.

Findings

Strictly dominates traditional simultaneous inference in power.

Nearly matches uncorrected intervals when only one question is plausible.

More applicable in nonparametric settings with better numerical stability.

Abstract

Selective inference is the problem of giving valid answers to statistical questions chosen in a data-driven manner. A standard solution to selective inference is simultaneous inference, which delivers valid answers to the set of all questions that could possibly have been asked. However, simultaneous inference can be unnecessarily conservative if this set includes many questions that were unlikely to be asked in the first place. We introduce a less conservative solution to selective inference that we call locally simultaneous inference, which only answers those questions that could plausibly have been asked in light of the observed data, all the while preserving rigorous type I error guarantees. For example, if the objective is to construct a confidence interval for the "winning" treatment effect in a clinical trial with multiple treatments, and it is obvious in hindsight that only one treatment had a chance to win, then our approach will return an interval that is nearly the same as the uncorrected, standard interval. Locally simultaneous inference is implemented by refining any method for simultaneous inference of interest. Under mild conditions satisfied by common confidence intervals, locally simultaneous inference strictly dominates its underlying simultaneous inference method, meaning it can never yield less statistical power but only more. Compared to conditional selective inference, which demands stronger guarantees, locally simultaneous inference is more easily applicable in nonparametric settings and is more numerically stable.