A Kernel Nonconformity Score for Multivariate Conformal Prediction

TL;DR

This paper introduces a multivariate kernel score for conformal prediction that adapts to geometric residual structures, providing finite-sample coverage guarantees and reducing prediction region volume.

Contribution

The paper proposes a novel multivariate kernel score that unifies Bayesian and frequentist uncertainty quantification, with dimension-free convergence rates and improved prediction regions.

Findings

Reduces prediction region volume significantly compared to ellipsoidal baselines.

Maintains nominal coverage with tighter regions, especially in higher dimensions.

Provides finite-sample coverage guarantees and convergence rates based on effective rank.

Abstract

Multivariate conformal prediction requires nonconformity scores that compress residual vectors into scalars while preserving certain implicit geometric structure of the residual distribution. We introduce a Multivariate Kernel Score (MKS) that produces prediction regions that explicitly adapt to this geometry. We show that the proposed score resembles the Gaussian process posterior variance, unifying Bayesian uncertainty quantification with the coverage guarantees of frequentist-type. Moreover, the MKS can be decomposed into an anisotropic Maximum Mean Discrepancy (MMD) that interpolates between kernel density estimation and covariance-weighted distance. We prove finite-sample coverage guarantees and establish convergence rates that depend on the effective rank of the kernel-based covariance operator rather than the ambient dimension, enabling dimension-free adaptation. On regression tasks, the MKS reduces the volume of prediction region significantly, compared to ellipsoidal baselines while maintaining nominal coverage, with larger gains at higher dimensions and tighter coverage levels.