IGT-OMD: Implicit Gradient Transport for Decision-Focused Learning under Delayed Feedback

TL;DR

This paper introduces IGT-OMD, a novel algorithm that reduces regret in decision-focused learning with delayed feedback by applying implicit gradient transport within online mirror descent.

Contribution

It provides the first sublinear regret bound for delayed bilevel optimization and demonstrates how implicit gradient transport mitigates staleness effects.

Findings

Transport error reduces from quadratic to linear with delay.

IGT-OMD achieves 17-55% decision loss reduction over baselines.

Transport benefit increases with delay, confirming theoretical predictions.

Abstract

Decision-focused learning trains predictive models end-to-end against downstream decision loss, but online settings suffer delayed feedback: outcomes may not arrive for many environment interactions. We identify \emph{staleness amplification}, a failure mode unique to bilevel optimization under delay, in which gradient staleness couples with inner-solver sensitivity to inflate regret beyond single-level delay theory. We prove that any black-box delayed optimizer incurs an irreducible regret cost from inner-solver approximation error, and that gradient staleness contributes a quadratically growing transport error without bilevel-aware correction. Our algorithm, \textbf{IGT-OMD}, applies Implicit Gradient Transport to hypergradients within Online Mirror Descent, re-evaluating stale gradients at the current parameters using stored inner solutions. This method reduces transport error from a quadratic to a linear dependence on delay and achieves the first sublinear regret bound for delayed bilevel optimization with queue-length-adaptive step sizes. Controlled experiments provide a \emph{mechanistic fingerprint}: transport benefit is exactly $0.0\%$ ($p=1.00$) at unit delay and grows monotonically to $9.5\%$ at fifty rounds ($p<0.001$), isolating the correction's effect. On Linear Quadratic Regulator, Warcraft shortest-path, and Sinkhorn optimal transport, IGT-OMD reduces decision loss by $17$--$55\%$ relative to single-level baselines, with phase transitions matching the theory.