Sparse Activations as Conformal Predictors

TL;DR

This paper introduces a novel approach combining conformal prediction with sparse softmax-like transformations, enabling efficient uncertainty quantification with coverage guarantees in classification tasks.

Contribution

It uncovers a new connection between conformal prediction and sparse transformations, proposing calibration methods that improve uncertainty sets in classification.

Findings

Achieves competitive coverage and efficiency in computer vision and text classification.

Supports adaptive and sparse prediction sets with coverage guarantees.

Demonstrates the effectiveness of the method compared to standard softmax-based approaches.

Abstract

Conformal prediction is a distribution-free framework for uncertainty quantification that replaces point predictions with sets, offering marginal coverage guarantees (i.e., ensuring that the prediction sets contain the true label with a specified probability, in expectation). In this paper, we uncover a novel connection between conformal prediction and sparse softmax-like transformations, such as sparsemax and $\gamma$-entmax (with $\gamma > 1$), which may assign nonzero probability only to a subset of labels. We introduce new non-conformity scores for classification that make the calibration process correspond to the widely used temperature scaling method. At test time, applying these sparse transformations with the calibrated temperature leads to a support set (i.e., the set of labels with nonzero probability) that automatically inherits the coverage guarantees of conformal prediction. Through experiments on computer vision and text classification benchmarks, we demonstrate that the proposed method achieves competitive results in terms of coverage, efficiency, and adaptiveness compared to standard non-conformity scores based on softmax.