Conformal Prediction Under Covariate Shift

TL;DR

This paper extends conformal prediction to handle covariate shift by using a weighted approach that accounts for distribution differences between training and test data, enabling distribution-free prediction intervals.

Contribution

It introduces a weighted conformal prediction method applicable under covariate shift with known or estimable likelihood ratios, broadening the methodology's applicability.

Findings

Valid prediction intervals under covariate shift

Applicable with estimated likelihood ratios from unlabeled data

Extends to weighted exchangeability settings

Abstract

We extend conformal prediction methodology beyond the case of exchangeable data. In particular, we show that a weighted version of conformal prediction can be used to compute distribution-free prediction intervals for problems in which the test and training covariate distributions differ, but the likelihood ratio between these two distributions is known---or, in practice, can be estimated accurately with access to a large set of unlabeled data (test covariate points). Our weighted extension of conformal prediction also applies more generally, to settings in which the data satisfies a certain weighted notion of exchangeability. We discuss other potential applications of our new conformal methodology, including latent variable and missing data problems.