From Conformal Predictions to Confidence Regions

TL;DR

This paper introduces CCR, a new conformal prediction-based method for constructing finite-sample valid confidence regions for model parameters, applicable to various models including linear ones, with empirical validation under complex noise conditions.

Contribution

The paper proposes CCR, a novel approach that combines conformal prediction intervals to create confidence regions for model parameters with minimal assumptions and broad applicability.

Findings

CCR provides finite-sample valid confidence regions.

Applicable to split, full, and cross-conformal methods.

Empirical results show robustness under heteroskedastic and non-Gaussian noise.

Abstract

Conformal prediction methodologies have significantly advanced the quantification of uncertainties in predictive models. Yet, the construction of confidence regions for model parameters presents a notable challenge, often necessitating stringent assumptions regarding data distribution or merely providing asymptotic guarantees. We introduce a novel approach termed CCR, which employs a combination of conformal prediction intervals for the model outputs to establish confidence regions for model parameters. We present coverage guarantees under minimal assumptions on noise and that is valid in finite sample regime. Our approach is applicable to both split conformal predictions and black-box methodologies including full or cross-conformal approaches. In the specific case of linear models, the derived confidence region manifests as the feasible set of a Mixed-Integer Linear Program (MILP), facilitating the deduction of confidence intervals for individual parameters and enabling robust optimization. We empirically compare CCR to recent advancements in challenging settings such as with heteroskedastic and non-Gaussian noise.