Weighted Aggregation of Conformity Scores for Classification

TL;DR

This paper introduces a method to combine multiple conformity scores in conformal prediction for multi-class classification, optimizing weights to produce smaller prediction sets while maintaining coverage guarantees.

Contribution

It proposes a novel weighted aggregation approach for conformity scores, supported by theoretical analysis and empirical validation showing improved efficiency over traditional single-score methods.

Findings

Outperforms single-score conformal predictors in prediction set size

Maintains valid coverage guarantees in classification tasks

Provides a theoretical link to Vapnik-Chervonenkis theory

Abstract

Conformal prediction is a powerful framework for constructing prediction sets with valid coverage guarantees in multi-class classification. However, existing methods often rely on a single score function, which can limit their efficiency and informativeness. We propose a novel approach that combines multiple score functions to improve the performance of conformal predictors by identifying optimal weights that minimize prediction set size. Our theoretical analysis establishes a connection between the weighted score functions and subgraph classes of functions studied in Vapnik-Chervonenkis theory, providing a rigorous mathematical basis for understanding the effectiveness of the proposed method. Experiments demonstrate that our approach consistently outperforms single-score conformal predictors while maintaining valid coverage, offering a principled and data-driven way to enhance the efficiency and practicality of conformal prediction in classification tasks.