Stochastic Optimization Forests

TL;DR

This paper introduces a novel method for training decision forests tailored for stochastic optimization problems, optimizing decision quality directly rather than prediction accuracy, and demonstrates its efficiency and effectiveness.

Contribution

The paper develops an optimization-aware forest training algorithm with approximate splitting criteria that scale efficiently and improve decision-making in stochastic settings.

Findings

Method reduces training time by up to 100 times.

Achieves asymptotic optimality in decision policies.

Performs close to exact re-optimization methods in experiments.

Abstract

We study contextual stochastic optimization problems, where we leverage rich auxiliary observations (e.g., product characteristics) to improve decision making with uncertain variables (e.g., demand). We show how to train forest decision policies for this problem by growing trees that choose splits to directly optimize the downstream decision quality, rather than splitting to improve prediction accuracy as in the standard random forest algorithm. We realize this seemingly computationally intractable problem by developing approximate splitting criteria that utilize optimization perturbation analysis to eschew burdensome re-optimization for every candidate split, so that our method scales to large-scale problems. We prove that our splitting criteria consistently approximate the true risk and that our method achieves asymptotic optimality. We extensively validate our method empirically, demonstrating the value of optimization-aware construction of forests and the success of our efficient approximations. We show that our approximate splitting criteria can reduce running time hundredfold, while achieving performance close to forest algorithms that exactly re-optimize for every candidate split.