Optimal Model Selection for Conformalized Robust Optimization

TL;DR

This paper introduces a new framework for model selection in Conformalized Robust Optimization, improving decision risk minimization and robustness control in decision-making under uncertainty.

Contribution

It proposes CROMS and its variants, E-CROMS, F-CROMS, and J-CROMS, for efficient and robust model selection in CRO, including covariate-aware methods.

Findings

Significant improvements in decision efficiency demonstrated in experiments.

The proposed algorithms achieve asymptotic robustness control.

Extensions to individualized settings enable covariate-aware model selection.

Abstract

In decision-making under uncertainty, Contextual Robust Optimization (CRO) provides reliability by minimizing the worst-case decision loss over a prediction set. While recent advances use conformal prediction to construct prediction sets for machine learning models, the downstream decisions critically depend on model selection. This paper introduces novel model selection frameworks for CRO that unify robustness control with decision risk minimization. We first propose Conformalized Robust Optimization with Model Selection (CROMS), a framework that selects the model to approximately minimize the averaged decision risk in CRO solutions. Given the target robustness level 1-\alpha, we present a computationally efficient algorithm called E-CROMS, which achieves asymptotic robustness control and decision optimality. To correct the control bias in finite samples, we further develop two algorithms: F-CROMS, which ensures a 1-\alpha robustness but requires searching the label space; and J-CROMS, which offers lower computational cost while achieving a 1-2\alpha robustness. Furthermore, we extend the CROMS framework to the individualized setting, where model selection is performed by minimizing the conditional decision risk given the covariates of the test data. This framework advances conformal prediction methodology by enabling covariate-aware model selection. Numerical results demonstrate significant improvements in decision efficiency across diverse synthetic and real-world applications, outperforming baseline approaches.