Gradient boosting for convex cone predict and optimize problems

TL;DR

This paper introduces dboost, a novel gradient boosting framework designed for predict-then-optimize problems, effectively reducing decision regret by integrating optimization into the training process.

Contribution

The paper presents the first general-purpose implementation of smart gradient boosting tailored for convex cone predict-then-optimize problems, supporting quadratic cone programming.

Findings

dboost outperforms existing SPO methods in reducing decision regret

Effective implicit differentiation enables gradient boosting in convex cone problems

Demonstrates improved decision quality in experiments

Abstract

Prediction models are typically optimized independently from decision optimization. A smart predict then optimize (SPO) framework optimizes prediction models to minimize downstream decision regret. In this paper we present dboost, the first general purpose implementation of smart gradient boosting for `predict, then optimize' problems. The framework supports convex quadratic cone programming and gradient boosting is performed by implicit differentiation of a custom fixed-point mapping. Experiments comparing with state-of-the-art SPO methods show that dboost can further reduce out-of-sample decision regret.