A Unified Comparative Study with Generalized Conformity Scores for Multi-Output Conformal Regression

TL;DR

This paper compares nine multi-output conformal prediction methods, introduces two new conformity scores ensuring coverage guarantees, and empirically evaluates their performance across diverse datasets.

Contribution

It provides a unified framework for multi-output conformal prediction, introduces novel conformity scores, and offers a comprehensive empirical comparison.

Findings

New conformity scores guarantee coverage in multi-output settings.

Broad applicability of proposed scores with generative models.

Empirical evaluation across 13 datasets demonstrates method performance.

Abstract

Conformal prediction provides a powerful framework for constructing distribution-free prediction regions with finite-sample coverage guarantees. While extensively studied in univariate settings, its extension to multi-output problems presents additional challenges, including complex output dependencies and high computational costs, and remains relatively underexplored. In this work, we present a unified comparative study of nine conformal methods with different multivariate base models for constructing multivariate prediction regions within the same framework. This study highlights their key properties while also exploring the connections between them. Additionally, we introduce two novel classes of conformity scores for multi-output regression that generalize their univariate counterparts. These scores ensure asymptotic conditional coverage while maintaining exact finite-sample marginal coverage. One class is compatible with any generative model, offering broad applicability, while the other is computationally efficient, leveraging the properties of invertible generative models. Finally, we conduct a comprehensive empirical evaluation across 13 tabular datasets, comparing all the multi-output conformal methods explored in this work. To ensure a fair and consistent comparison, all methods are implemented within a unified code base.