Online Decision-Focused Learning

TL;DR

This paper develops online algorithms for decision-focused learning in dynamic environments, addressing challenges of non-convexity and non-differentiability, and provides theoretical regret bounds with empirical validation.

Contribution

It introduces the first provably guaranteed online algorithms for decision-focused learning in evolving environments, handling non-convex and non-differentiable objectives.

Findings

Algorithms outperform benchmarks in knapsack experiments

Established static and dynamic regret bounds

Effective handling of non-convex, non-differentiable objectives

Abstract

Decision-focused learning (DFL) is an increasingly popular paradigm for training predictive models whose outputs are used in decision-making tasks. Instead of merely optimizing for predictive accuracy, DFL trains models to directly minimize the loss associated with downstream decisions. However, existing studies focus solely on scenarios where a fixed batch of data is available and the objective function does not change over time. We instead investigate DFL in dynamic environments where the objective function and data distribution evolve over time. This setting is challenging for online learning because the objective function has zero or undefined gradients, which prevents the use of standard first-order optimization methods, and is generally non-convex. To address these difficulties, we (i) regularize the objective to make it differentiable and (ii) use perturbation techniques along with a near-optimal oracle to overcome non-convexity. Combining those techniques yields two original online algorithms tailored for DFL, for which we establish respectively static and dynamic regret bounds. These are the first provable guarantees for the online decision-focused problem. Finally, we showcase the effectiveness of our algorithms on a knapsack experiment, where they outperform two standard benchmarks.