A data-driven approach to beating SAA out-of-sample

TL;DR

This paper introduces Distributionally Optimistic Optimization (DOO) models that can outperform SAA out-of-sample, but with increased sensitivity and reduced robustness, highlighting a trade-off between optimism and stability.

Contribution

The paper proposes DOO models that guarantee outperforming SAA out-of-sample and analyzes their robustness and calibration challenges.

Findings

DOO models can outperform SAA out-of-sample.

Optimistic solutions are more sensitive to model error.

Calibrating models for outperforming SAA is challenging with limited data.

Abstract

While solutions of Distributionally Robust Optimization (DRO) problems can sometimes have a higher out-of-sample expected reward than the Sample Average Approximation (SAA), there is no guarantee. In this paper, we introduce a class of Distributionally Optimistic Optimization (DOO) models, and show that it is always possible to ``beat" SAA out-of-sample if we consider not just worst-case (DRO) models but also best-case (DOO) ones. We also show, however, that this comes at a cost: Optimistic solutions are more sensitive to model error than either worst-case or SAA optimizers, and hence are less robust and calibrating the worst- or best-case model to outperform SAA may be difficult when data is limited.