Is Inference Conditional on Not Rejecting a Pre-test Less Reliable than Unconditional Inference?

TL;DR

This paper investigates whether conducting inference only after a pre-test affects its validity, showing that it remains valid under certain conditions but can be conservative or have coverage issues otherwise.

Contribution

It provides theoretical results demonstrating the validity and limitations of pre-test conditional inference under various regularity conditions.

Findings

Conditional inference remains valid if tested conditions hold, though often conservative.

When conditions do not hold, confidence intervals can have larger conditional coverage than unconditional.

Validity is unaffected by the dependence between the estimator and the pre-test.

Abstract

Assume that an estimator is asymptotically normal for a target parameter under some conditions. Suppose also that one can test these conditions, and one conducts inference for the target only if the pre-test is not rejected. Does such pre-testing undermine inference? We show that if the tested conditions and mild regularity restrictions hold, conditional inference is still valid, albeit typically conservative. Validity holds regardless of the asymptotic dependence between the estimator and the pre-test. If the tested conditions do not hold, we exhibit conditions under which confidence intervals have larger conditional than unconditional coverage.