Pessimistic bilevel optimization approach for decision-focused learning

TL;DR

This paper introduces a pessimistic bilevel optimization method for decision-focused learning in combinatorial problems, demonstrating improved out-of-sample regret over existing approaches through specialized algorithms and benchmarking.

Contribution

It proposes a novel pessimistic bilevel approach with a cut generation algorithm for decision-focused learning, addressing limitations of existing methods.

Findings

Reduces out-of-sample regret compared to SPO+ and estimate-then-optimize.

Effective on 0-1 knapsack problem benchmarks.

Demonstrates advantages in computational experiments.

Abstract

The recent interest in contextual optimization problems, where randomness is associated with side information, has led to two primary strategies for formulation and solution. The first, estimate-then-optimize, separates the estimation of the problem's parameters from the optimization process. The second, decision-focused optimization, integrates the optimization problem's structure directly into the prediction procedure. In this work, we propose a pessimistic bilevel approach for solving general decision-focused formulations of combinatorial optimization problems. Our method solves an $\varepsilon$-approximation of the pessimistic bilevel problem using a specialized cut generation algorithm. We benchmark its performance on the 0-1 knapsack problem against estimate-then-optimize and decision-focused methods, including the popular SPO+ approach. Computational experiments highlight the proposed method's advantages, particularly in reducing out-of-sample regret.