Symmetric Aggregation of Conformity Scores for Efficient Uncertainty Sets

TL;DR

This paper introduces SACP, a novel symmetric aggregation method for conformal prediction that combines multiple models' uncertainty scores to produce more reliable and sharper prediction sets, supported by theoretical and experimental validation.

Contribution

The paper proposes SACP, a flexible, symmetric aggregation framework for conformal prediction that improves uncertainty quantification by effectively combining multiple models' scores.

Findings

SACP consistently improves the efficiency of prediction sets.

SACP outperforms existing model aggregation baselines.

Theoretical analysis supports the validity of the approach.

Abstract

Access to multiple predictive models trained for the same task, whether in regression or classification, is increasingly common in many applications. Aggregating their predictive uncertainties to produce reliable and efficient uncertainty quantification is therefore a critical but still underexplored challenge, especially within the framework of conformal prediction (CP). While CP methods can generate individual prediction sets from each model, combining them into a single, more informative set remains a challenging problem. To address this, we propose SACP (Symmetric Aggregated Conformal Prediction), a novel method that aggregates nonconformity scores from multiple predictors. SACP transforms these scores into e-values and combines them using any symmetric aggregation function. This flexible design enables a robust, data-driven framework for selecting aggregation strategies that yield sharper prediction sets. We also provide theoretical insights that help justify the validity and performance of the SACP approach. Extensive experiments on diverse datasets show that SACP consistently improves efficiency and often outperforms state-of-the-art model aggregation baselines.