Unveil Sources of Uncertainty: Feature Contribution to Conformal Prediction Intervals

TL;DR

This paper introduces a model-agnostic method for attributing predictive uncertainty in machine learning models using conformal prediction and cooperative game theory, providing deeper interpretability of uncertainty sources.

Contribution

It proposes a novel uncertainty attribution method based on conformal prediction and Harsanyi allocations, extending explainability to predictive uncertainty.

Findings

Effective uncertainty attribution demonstrated on synthetic and real datasets

Improved computational efficiency with Monte Carlo approximation

Enhanced interpretability of model uncertainty in high-stakes applications

Abstract

Cooperative game theory methods, notably Shapley values, have significantly enhanced machine learning (ML) interpretability. However, existing explainable AI (XAI) frameworks mainly attribute average model predictions, overlooking predictive uncertainty. This work addresses that gap by proposing a novel, model-agnostic uncertainty attribution (UA) method grounded in conformal prediction (CP). By defining cooperative games where CP interval properties-such as width and bounds-serve as value functions, we systematically attribute predictive uncertainty to input features. Extending beyond the traditional Shapley values, we use the richer class of Harsanyi allocations, and in particular the proportional Shapley values, which distribute attribution proportionally to feature importance. We propose a Monte Carlo approximation method with robust statistical guarantees to address computational feasibility, significantly improving runtime efficiency. Our comprehensive experiments on synthetic benchmarks and real-world datasets demonstrate the practical utility and interpretative depth of our approach. By combining cooperative game theory and conformal prediction, we offer a rigorous, flexible toolkit for understanding and communicating predictive uncertainty in high-stakes ML applications.