TL;DR
This paper introduces an adaptive conformal risk control method that dynamically adjusts to the difficulty of individual test samples, providing more reliable and fine-grained performance guarantees for black-box machine learning models.
Contribution
It proposes a novel data-driven framework for approximate conditional risk control that adapts to sample difficulty, extending beyond traditional conditioning methods.
Findings
Enhanced risk control in regression and segmentation tasks.
Superior precision over traditional risk-control methods.
Effective adaptation to sample difficulty in real-time.
Abstract
Science and technology have a growing need for effective mechanisms that ensure reliable, controlled performance from black-box machine learning algorithms. These performance guarantees should ideally hold conditionally on the input-that is the performance guarantees should hold, at least approximately, no matter what the input. However, beyond stylized discrete groupings such as ethnicity and gender, the right notion of conditioning can be difficult to define. For example, in problems such as image segmentation, we want the uncertainty to reflect the intrinsic difficulty of the test sample, but this may be difficult to capture via a conditioning event. Building on the recent work of Gibbs et al. [2023], we propose a methodology for achieving approximate conditional control of statistical risks-the expected value of loss functions-by adapting to the difficulty of test samples. Our…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
