Aggregating Conformal Prediction Sets via {\alpha}-Allocation

TL;DR

This paper introduces COLA, a novel aggregation method for conformal prediction sets that optimally allocates confidence levels across multiple scores, resulting in smaller prediction sets with guaranteed coverage.

Contribution

The work proposes a new aggregation strategy, COLA, with variants that ensure finite-sample coverage and improved set efficiency, advancing conformal prediction methods.

Findings

COLA produces smaller prediction sets than existing methods.

COLA variants guarantee finite-sample marginal coverage.

Experimental results show COLA's superior performance on synthetic and real data.

Abstract

Conformal prediction offers a distribution-free framework for constructing prediction sets with finite-sample coverage. Yet, efficiently leveraging multiple conformity scores to reduce prediction set size remains a major open challenge. Instead of selecting a single best score, this work introduces a principled aggregation strategy, COnfidence-Level Allocation (COLA), that optimally allocates confidence levels across multiple conformal prediction sets to minimize empirical set size while maintaining provable coverage. Two variants are further developed, COLA-s and COLA-f, which guarantee finite-sample marginal coverage via sample splitting and full conformalization, respectively. In addition, we develop COLA-l, an individualized allocation strategy that promotes local size efficiency while achieving asymptotic conditional coverage. Extensive experiments on synthetic and real-world datasets demonstrate that COLA achieves considerably smaller prediction sets than state-of-the-art baselines while maintaining valid coverage.