Data-Driven Sample Average Approximation with Covariate Information

TL;DR

This paper develops new data-driven stochastic programming methods that incorporate machine learning predictions with covariate information to improve decision-making under uncertainty, providing theoretical guarantees and empirical validation.

Contribution

It introduces two novel SAA frameworks using out-of-sample residuals for scenario generation, accommodating various regression techniques, with proven consistency and asymptotic optimality.

Findings

New frameworks outperform existing methods in limited data scenarios.

Theoretical guarantees include convergence rates and finite-sample bounds.

Computational experiments confirm the advantages of the proposed methods.

Abstract

We study optimization for data-driven decision-making when we have observations of the uncertain parameters within the optimization model together with concurrent observations of covariates. Given a new covariate observation, the goal is to choose a decision that minimizes the expected cost conditioned on this observation. We investigate three data-driven frameworks that integrate a machine learning prediction model within a stochastic programming sample average approximation (SAA) for approximating the solution to this problem. Two of the SAA frameworks are new and use out-of-sample residuals of leave-one-out prediction models for scenario generation. The frameworks we investigate are flexible and accommodate parametric, nonparametric, and semiparametric regression techniques. We derive conditions on the data generation process, the prediction model, and the stochastic program under which solutions of these data-driven SAAs are consistent and asymptotically optimal, and also derive convergence rates and finite sample guarantees. Computational experiments validate our theoretical results, demonstrate the potential advantages of our data-driven formulations over existing approaches (even when the prediction model is misspecified), and illustrate the benefits of our new data-driven formulations in the limited data regime.