Leverage-Weighted Conformal Prediction

TL;DR

Leverage-Weighted Conformal Prediction (LWCP) introduces a novel, geometry-based weighting scheme to improve adaptive prediction intervals, achieving better conditional coverage without auxiliary models or hyperparameters.

Contribution

LWCP offers a new leverage-based weighting method for conformal prediction that guarantees validity and improves conditional coverage adaptively, with minimal computational overhead.

Findings

LWCP preserves finite-sample marginal validity.

LWCP achieves asymptotically optimal conditional coverage.

Experiments show reduced coverage disparity in real and synthetic data.

Abstract

Split conformal prediction provides distribution-free prediction intervals with finite-sample marginal coverage, but produces constant-width intervals that overcover in low-variance regions and undercover in high-variance regions. Existing adaptive methods require training auxiliary models. We propose Leverage-Weighted Conformal Prediction (LWCP), which weights nonconformity scores by a function of the statistical leverage -- the diagonal of the hat matrix -- deriving adaptivity from the geometry of the design matrix rather than from auxiliary model fitting. We prove that LWCP preserves finite-sample marginal validity for any weight function; achieves asymptotically optimal conditional coverage at essentially no width cost when heteroscedasticity factors through leverage; and recovers the form and width of classical prediction intervals under Gaussian assumptions while retaining distribution-free guarantees. We further establish that randomized leverage approximations preserve coverage exactly with controlled width perturbation, and that vanilla CP suffers a persistent, sample-size-independent conditional coverage gap that LWCP eliminates. The method requires no hyperparameters beyond the choice of weight function and adds negligible computational overhead to vanilla CP. Experiments on synthetic and real data confirm the theoretical predictions, demonstrating substantial reductions in conditional coverage disparity across settings.