Provably Data-driven Lagrangian Relaxation for Mixed Integer Linear Programming

TL;DR

This paper provides a theoretical analysis of data-driven Lagrangian Relaxation for MILP, establishing bounds and optimal rates for learning multipliers and extending to warm-start scenarios.

Contribution

It derives generalization bounds, proves minimax lower bounds, and shows that stochastic gradient ascent achieves optimal rates for learning multipliers in MILP.

Findings

Derived a generalization bound of $\\mathcal{O}(s^{1.5}/\rac{1}{2}\sqrt{N})$ for learned multipliers.

Proved a minimax lower bound of $\\\Omega(s/\rac{1}{2}\sqrt{N})$, showing linear dependency is unavoidable.

Showed that stochastic gradient ascent with averaging achieves the minimax optimal rate $\\Theta(s/\\sqrt{N})$.

Abstract

Lagrangian Relaxation (LR) is a powerful technique for solving large-scale Mixed Integer Linear Programming (MILP), particularly those with decomposable structures, such as vehicle routing or unit commitment problems. By relaxing the coupling constraints, LR enables parallel subproblem solving and often yields tighter dual bounds than standard linear programming relaxations, which is crucial for efficient branch-and-bound pruning. While recent empirical work has shown promising results using machine learning to predict these multipliers, a theoretical understanding of such methods remains an open question. In this work, we bridge this gap by analyzing the problem of learning LR through the lens of Data-driven Algorithm Design, i.e., a statistical learning problem over a distribution of problem instances. Our contributions are as follows: first, we derive a generalization bound of $\mathcal{O}(s^{1.5}/\sqrt{N})$ for the learned multipliers, where $s$ is the number of coupling constraints and $N$ is the sample size. Second, we provide a minimax lower-bound of $\Omega(s/\sqrt{N})$, proving that a linear dependency is unavoidable. Third, we constructively close this theoretical gap by proving that Stochastic Gradient Ascent (SGA) with averaging achieves the minimax optimal rate $\Theta(s/\sqrt{N})$. Finally, we extend our framework to the learning-to-warm-start setting, proving that it achieves a fast, minimax-optimal rate of $\Theta(s/N)$ and establishing a theoretical advantage over direct multiplier prediction.