Conformalized Percentile Interval: Finite Sample Validity and Improved Conditional Performance

TL;DR

This paper introduces a conformal prediction method using neural network-estimated conditional CDFs and PIT calibration to achieve finite-sample validity and improved conditional performance.

Contribution

It proposes a novel calibration approach in PIT space that enhances conditional calibration and interval efficiency in complex predictive settings.

Findings

Finite-sample marginal coverage is guaranteed.

Method achieves asymptotic conditional coverage under mild conditions.

Experiments show shorter, better-calibrated intervals than existing methods.

Abstract

Conformal prediction provides distribution-free predictive intervals with finite-sample marginal coverage. However, achieving conditional validity and interval efficiency (in terms of short interval length) remains challenging, particularly in complex settings with heteroskedasticity, skewed responses, or estimation errors. We propose a conformal-style calibration method for responses obtained by the probability integral transform (PIT) of the conditional cumulative distribution function (CDF) estimated via neural networks to construct a finite-sample-adjusted percentile interval with the shortest length determined by the estimated conditional CDF. Calibrating in PIT space is effective because PIT values are asymptotically feature-independent when the CDF estimator is accurate, which mitigates feature-dependent miscoverage and improves conditional calibration. On the other hand, our percentile calibration adapts to the empirical PIT distribution, which is robust against a possibly imperfect estimation of the conditional CDF. We prove the finite-sample marginal coverage property of the proposed method and show its asymptotic conditional coverage under mild consistency conditions. Experiments on diverse synthetic and real-world benchmarks demonstrate better conditional calibration and substantially shorter intervals than existing methods.