Optimized Certainty Equivalent Risk-Controlling Prediction Sets

TL;DR

This paper introduces OCE-RCPS, a new framework for creating prediction sets with high-probability guarantees on complex risk measures, improving reliability in safety-critical applications like medical imaging.

Contribution

It develops a novel OCE-RCPS method that offers probabilistic guarantees on advanced risk measures, addressing limitations of existing risk-controlling prediction sets.

Findings

OCE-RCPS satisfies probabilistic risk constraints in experiments.

OCE-RCPS outperforms existing methods in reliability guarantees.

Experiments demonstrate consistent risk control across various settings.

Abstract

In safety-critical applications such as medical image segmentation, prediction systems must provide reliability guarantees that extend beyond conventional expected loss control. While risk-controlling prediction sets (RCPS) offer probabilistic guarantees on the expected risk, they fail to capture tail behavior and worst-case scenarios that are crucial in high-stakes settings. This paper introduces optimized certainty equivalent RCPS (OCE-RCPS), a novel framework that provides high-probability guarantees on general optimized certainty equivalent (OCE) risk measures, including conditional value-at-risk (CVaR) and entropic risk. OCE-RCPS leverages upper confidence bounds to identify prediction set parameters that satisfy user-specified risk tolerance levels with provable reliability. We establish theoretical guarantees showing that OCE-RCPS satisfies the desired probabilistic constraint for loss functions such as miscoverage and false negative rate. Experiments on image segmentation demonstrate that OCE-RCPS consistently meets target satisfaction rates across various risk measures and reliability configurations, while OCE-CRC fails to provide probabilistic guarantees.