Two sources of poor coverage of confidence intervals after model selection

TL;DR

This paper investigates two main reasons for the poor coverage of confidence intervals after model selection: incorrect model choice and data reuse in interval construction, highlighting their impact on statistical inference accuracy.

Contribution

The study compares the effects of model selection errors and data reuse on confidence interval coverage, providing insights into their relative importance and implications.

Findings

Incorrect model selection leads to undercoverage of confidence intervals.

Data reuse in interval construction exacerbates coverage issues.

Both factors significantly impact the reliability of post-model-selection inference.

Abstract

We compare the following two sources of poor coverage of post-model-selection confidence intervals: the preliminary data-based model selection sometimes chooses the wrong model and the data used to choose the model is re-used for the construction of the confidence interval.