Decision-Focused Learning via Tangent-Space Projection of Prediction Error

TL;DR

This paper introduces PEAR, a novel method for decision-focused learning that efficiently computes regret gradients by projecting prediction errors onto the tangent space of active constraints, improving decision quality and computational efficiency.

Contribution

The paper provides a geometric characterization of regret gradients and proposes PEAR, a new approach that avoids differentiating through solvers, enhancing efficiency and decision accuracy.

Findings

PEAR achieves superior decision quality compared to baselines.

PEAR is more computationally efficient than existing methods.

Gains persist under constraint shifts.

Abstract

Decision-Focused Learning (DFL) trains predictors to improve downstream decision quality, but computing regret gradients typically requires differentiating through solvers or relying on surrogate losses, which can be computationally expensive or deviate from the true objective. We show that, under standard regularity with locally stable active constraints, the regret gradient admits a closed-form geometric characterization, equivalent to the prediction error projected onto the tangent space of active constraints, scaled by local curvature. This reveals that regret gradients can be obtained by filtering decision-irrelevant components from the MSE gradient, providing a simpler and more direct alternative to existing approaches. Based on this, we propose PEAR (Projected Error As Regret-gradient), which computes regret gradients via a reduced linear system over active constraints, avoiding differentiation through solver iterations or additional optimization solves. Experiments on LP benchmarks and a real-world QP task show that PEAR achieves the best decision quality among all baselines while being the most computationally efficient, with gains that persist under constraint shifts.