Enhanced Multi-model Online Conformal Prediction

TL;DR

This paper introduces a multi-model online conformal prediction method that enhances efficiency and reduces computational costs by dynamically selecting effective models, leading to smaller prediction sets and better performance in online settings.

Contribution

It proposes a novel algorithm that uses bipartite graphs to select effective models, improving over existing methods in both prediction accuracy and computational efficiency.

Findings

Outperforms existing methods in prediction set size

Reduces computational complexity significantly

Achieves better online adaptation and efficiency

Abstract

Conformal prediction is a framework for uncertainty quantification that constructs prediction sets for previously unseen data, guaranteeing coverage of the true label with a specified probability. However, the efficiency of these prediction sets, measured by their size, depends on the choice of the underlying learning model. Relying on a single fixed model may lead to suboptimal performance in online environments, as a single model may not consistently perform well across all time steps. To mitigate this, prior work has explored selecting a model from a set of candidates. However, this approach becomes computationally expensive as the number of candidate models increases. Moreover, poorly performing models in the set may also hinder the effectiveness. To tackle this challenge, this work develops a novel multi-model online conformal prediction algorithm that reduces computational complexity and improves prediction efficiency. At each time step, a bipartite graph is generated to identify a subset of effective models, from which a model is selected to construct the prediction set. Experiments demonstrate that our method outperforms existing multi-model conformal prediction techniques in terms of both prediction set size and computational efficiency.