Scalable Decision-Focused Learning through Cost-Sensitive Regression

TL;DR

This paper introduces a cost-sensitive regression framework for decision-focused learning, significantly improving scalability and efficiency in solving combinatorial optimization problems with uncertain parameters.

Contribution

It formalizes a novel loss function framework that reduces computational costs by requiring minimal problem solves during training, enabling scalable decision-focused learning.

Findings

Achieves comparable downstream task quality to state-of-the-art methods.

Requires only zero or one solve per training instance, greatly improving efficiency.

Enables scaling to larger problem sizes previously infeasible with existing DFL methods.

Abstract

Many real-world combinatorial problems involve uncertain parameters, which can be predicted given contextual features and historical data. These `predict-then-optimize' or `contextual optimization' problems have gained significant attention: end-to-end training methods can now minimize the downstream task cost rather than the predictive error. However, despite their effectiveness, these decision-focused learning (DFL) approaches often rely on repeated solving of the underlying combinatorial optimization problem during training, making them computationally expensive and difficult to scale. We reframe the learning problem as a cost-sensitive multi-output regression problem: multi-output due to the combinatorial problem having multiple uncertain parameters, and cost-sensitive due to the downstream task cost being the real target. Our technical contribution is the formalization of multiple loss function components that follow from this reframing: cost-insensitive normalization, decision-aware asymmetric penalization of over- and underpredictions, and instance-based costs that mimic the true downstream task-based loss locally. These components require zero or one solve per training data instance, while requiring no further solves during training. Experiments show that the combination of loss components achieves comparable downstream task quality to the state of the art, while being significantly more efficient, enabling scaling to problem sizes that have not been tackled before with DFL.