Weight Clipping for Robust Conformal Inference under Unbounded Covariate Shifts

TL;DR

This paper introduces a new method called CLISF for density ratio estimation in conformal prediction, improving robustness under unbounded covariate shifts by providing theoretical guarantees and correction techniques.

Contribution

It proposes CLISF, a reduced-variance density ratio estimator, and demonstrates its effectiveness with theoretical guarantees for conformal inference under covariate shifts.

Findings

CLISF reduces variance in density ratio estimation.

WCP with CLISF achieves dataset-conditional coverage.

The method corrects undercoverage by data-driven inflation of coverage target.

Abstract

Conformal prediction (CP) provides powerful, distribution-free prediction sets, but its guarantees rely on the exchangeability of training and test data, which is often violated in practice due to covariate shifts. While weighted conformal prediction (WCP) is designed to handle such shifts, it can suffer from significant undercoverage when the density ratio between the distributions is unbounded and/or must be learned. This is because of both overfitting in learning the density ratio, and high variance in estimating the nonconformity score threshold. To address this, we introduce clipped least-squares importance fitting (CLISF) as a reduced-variance method for density ratio estimation. Specifically, we show that density ratios learned using CLISF, when plugged into WCP, have bounded expected undercoverage. Furthermore, we show that the undercoverage can be corrected by running WCP with a slightly inflated coverage target; crucially, we are able to estimate the required level of inflation from the data. We provide the first theoretical guarantees for weight clipping in conformal inference, achieving dataset-conditional coverage with a sample complexity that does not blow up with the higher moments of the true density ratio -- a key limitation of prior work. We verify our results on real-world benchmarks and synthetic data.