Flexible Selective Inference with Flow-based Transport Maps

TL;DR

This paper introduces a flow-based transport map approach to approximate complex conditional distributions for selective inference, enabling valid p-values and confidence sets even with intractable selection events.

Contribution

It proposes a novel method using normalizing flows to perform flexible selective inference without requiring analytical characterization of the selection event.

Findings

Provides valid p-values and confidence sets for selected hypotheses

Enables likelihood-based and quantile-based inference

Adjusts for intractable selection steps in complex procedures

Abstract

Data-carving methods perform selective inference by conditioning the distribution of data on the observed selection event. However, existing data-carving approaches typically require an analytically tractable characterization of the selection event. This paper introduces a new method that leverages tools from flow-based generative modeling to approximate a potentially complex conditional distribution, even when the underlying selection event lacks an analytical description -- take, for example, the data-adaptive tuning of model parameters. The key idea is to learn a transport map that pushes forward a simple reference distribution to the conditional distribution given selection. This map is efficiently learned via a normalizing flow, without imposing any further restrictions on the nature of the selection event. Through extensive numerical experiments on both simulated and real data, we demonstrate that this method enables flexible selective inference by providing: (i) valid p-values and confidence sets for adaptively selected hypotheses and parameters, (ii) a closed-form expression for the conditional density function, enabling likelihood-based and quantile-based inference, and (iii) adjustments for intractable selection steps that can be easily integrated with existing methods designed to account for the tractable steps in a selection procedure involving multiple steps.