C-Adapter: Adapting Deep Classifiers for Efficient Conformal Prediction Sets

TL;DR

This paper introduces C-Adapter, a novel adapter-based tuning method that improves the efficiency of conformal predictors in deep classifiers without reducing accuracy, by enhancing non-conformity score discrimination.

Contribution

The paper proposes C-Adapter, an intra order-preserving adapter that optimizes non-conformity scores, significantly improving conformal prediction efficiency while maintaining classifier accuracy.

Findings

C-Adapter improves prediction set efficiency across coverage rates.

It enhances conformal training methods.

It adapts various classifiers effectively.

Abstract

Conformal prediction, as an emerging uncertainty quantification technique, typically functions as post-hoc processing for the outputs of trained classifiers. To optimize the classifier for maximum predictive efficiency, Conformal Training rectifies the training objective with a regularization that minimizes the average prediction set size at a specific error rate. However, the regularization term inevitably deteriorates the classification accuracy and leads to suboptimal efficiency of conformal predictors. To address this issue, we introduce \textbf{Conformal Adapter} (C-Adapter), an adapter-based tuning method to enhance the efficiency of conformal predictors without sacrificing accuracy. In particular, we implement the adapter as a class of intra order-preserving functions and tune it with our proposed loss that maximizes the discriminability of non-conformity scores between correctly and randomly matched data-label pairs. Using C-Adapter, the model tends to produce extremely high non-conformity scores for incorrect labels, thereby enhancing the efficiency of prediction sets across different coverage rates. Extensive experiments demonstrate that C-Adapter can effectively adapt various classifiers for efficient prediction sets, as well as enhance the conformal training method.