Carving model-free inference

TL;DR

This paper extends the theoretical foundation of carving, a method for valid selective inference that reuses pilot data, to model-free settings, demonstrating its effectiveness through simulations and real data.

Contribution

It develops asymptotic theory to justify carving beyond Gaussian models, enabling broader application in model-free inference.

Findings

Carving produces valid confidence intervals in model-free settings.

Carving yields tighter intervals compared to traditional methods.

The method is effective in both simulated and real data scenarios.

Abstract

Complex studies involve many steps. Selecting promising findings based on pilot data is a first step. As more observations are collected, the investigator must decide how to combine the new data with the pilot data to construct valid selective inference. Carving, introduced by Fithian et al. (2014), enables the reuse of pilot data during selective inference and accounts for over-optimism from the selection process. Currently, the justification for carving is tied to parametric models, like the commonly used Gaussian model. In this paper, we develop the asymptotic theory to substantiate the use of carving beyond Gaussian models. Through both simulated and real instances, we find that carving produces valid and tight confidence intervals within a model-free setting.