Improving Adaptive Conformal Prediction Using Self-Supervised Learning

TL;DR

This paper explores how self-supervised learning can enhance conformal prediction by improving the adaptiveness and efficiency of prediction intervals, demonstrating empirical benefits on synthetic and real datasets.

Contribution

It introduces a method to incorporate self-supervised errors into conformal prediction to improve interval adaptiveness and efficiency.

Findings

Self-supervised errors improve conformal interval width.

Enhanced intervals show better coverage and adaptiveness.

Method effective on both synthetic and real data.

Abstract

Conformal prediction is a powerful distribution-free tool for uncertainty quantification, establishing valid prediction intervals with finite-sample guarantees. To produce valid intervals which are also adaptive to the difficulty of each instance, a common approach is to compute normalized nonconformity scores on a separate calibration set. Self-supervised learning has been effectively utilized in many domains to learn general representations for downstream predictors. However, the use of self-supervision beyond model pretraining and representation learning has been largely unexplored. In this work, we investigate how self-supervised pretext tasks can improve the quality of the conformal regressors, specifically by improving the adaptability of conformal intervals. We train an auxiliary model with a self-supervised pretext task on top of an existing predictive model and use the self-supervised error as an additional feature to estimate nonconformity scores. We empirically demonstrate the benefit of the additional information using both synthetic and real data on the efficiency (width), deficit, and excess of conformal prediction intervals.