The Bias-Variance Tradeoff in Data-Driven Optimization: A Local Misspecification Perspective

TL;DR

This paper analyzes the bias-variance tradeoff in data-driven optimization methods under local model misspecification, providing new theoretical insights into their relative performance and decision bias.

Contribution

It introduces a granular analysis of model-based and data-driven methods under local misspecification, revealing bias-variance tradeoffs and explicit bias expressions.

Findings

Bias-variance tradeoff depends on degree of local misspecification

Explicit formulas for decision bias under misspecification

Geometric insights into variance and impact of misspecification

Abstract

Data-driven stochastic optimization is ubiquitous in machine learning and operational decision-making problems. Sample average approximation (SAA) and model-based approaches such as estimate-then-optimize (ETO) or integrated estimation-optimization (IEO) are all popular, with model-based approaches being able to circumvent some of the issues with SAA in complex context-dependent problems. Yet the relative performance of these methods is poorly understood, with most results confined to the dichotomous cases of the model-based approach being either well-specified or misspecified. We develop the first results that allow for a more granular analysis of the relative performance of these methods under a local misspecification setting, which models the scenario where the model-based approach is nearly well-specified. By leveraging tools from contiguity theory in statistics, we show that there is a bias-variance tradeoff between SAA, IEO, and ETO under local misspecification, and that the relative importance of the bias and the variance depends on the degree of local misspecification. Moreover, we derive explicit expressions for the decision bias, which allows us to characterize (un)impactful misspecification directions, and provide further geometric understanding of the variance.