Conformalized Decision Risk Assessment
Wenbin Zhou, Agni Orfanoudaki, Shixiang Zhu
TL;DR
CREDO is a distribution-free framework that quantifies the probability of a decision remaining optimal under uncertainty using conformal prediction, applicable across diverse optimization problems.
Contribution
It introduces a novel, model-agnostic method for estimating decision risk with finite-sample guarantees using conformal prediction balls.
Findings
Provides accurate bounds on decision optimality probability
Demonstrates broad applicability across optimization problems
Offers efficient and reliable risk assessments
Abstract
In many operational settings, decision-makers must commit to actions before uncertainty resolves, but existing optimization tools rarely quantify how consistently a chosen decision remains optimal across plausible scenarios. This paper introduces CREDO -- Conformalized Risk Estimation for Decision Optimization, a distribution-free framework that quantifies the probability that a prescribed decision remains (near-)optimal across realizations of uncertainty. CREDO reformulates decision risk through the inverse feasible region -- the set of outcomes under which a decision is optimal -- and estimates its probability using inner approximations constructed from conformal prediction balls generated by a conditional generative model. This approach yields finite-sample, distribution-free lower bounds on the probability of decision optimality. The framework is model-agnostic and broadly…
Peer Reviews
Decision·ICLR 2026 Poster
- The problem of designing ML-powered, distribution-free valid risk certificates is a timely and impactful problem. The authors make a meaningful step by connecting this problem to running CP on an inverse optimization problem. - Their work has a very good balance of theory and experiment. The theoretical guarantees are of the interest of practice, and they showcase that it actually works in real world datasets.
My first concern is regarding "selection bias". Selection bias is well-known phenomena is statistics, which points toward the scenario where a decision maker wants to use an estimation to inform their decisions. The bias arises, when the estimation is performed without the knowledge of the down stream decision problem, and this can potentially disrupt the statistical guarantees of the original estimation. In the context of the problem that is studied in this paper, it shows itself as follows: sa
1. The paper addresses an under explored research area about how to provide rigorous, interpretable risk certificates for candidate decisions in high stakes, uncertain environments. 2. The "decide-then-assess" paradigm is a reasonable variation from the standard "predict-then-optimize" pipeline, and is well-motivated by practical needs for human-AI collaboration. 3. The use of inverse optimization geometry to characterize the optimality region for a decision is well done. 4. The integration of c
1. While the method is general, the closed form efficiency is only for linear programs. For nonlinear or combinatorial problems, the computational cost of characterizing the inverse feasible region may be significant. 2. The approach assumes access to a well calibrated conditional model for the uncertain parameters. While the paper uses generative models to estimate the conditional distribution, in practice, any model that can accurately capture and sample from P(Y∣X) would suffice, including pa
- The paper's motivation is practical. The interpretable risk assessment in high-stakes domains is widely applicable and important. - The theorems on conservativeness and the Monte Carlo interpretation demonstrate sound reasoning and attention to statistical guarantees. - The idea of using inverse optimal space of outcome to find an upper bound on distribution-free probability is refreshing and can lead to potentially stronger results.
- The computational efficiency of the framework is not discussed. Finding the inverse space of the outcome where a given decision is optimal can be NP-hard for any objective function. It is also NP-hard to check whether the conformal set is included by the inverse space. The radius assumption here is still not enough since the inverse space can be non-convex. - The upper bound that is found by the framework could be arbitrarily bad. That says there is no result on the lower bound of the probabil
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsRisk and Portfolio Optimization · Advanced Multi-Objective Optimization Algorithms · Gaussian Processes and Bayesian Inference