Sobolev Training of End-to-End Optimization Proxies

TL;DR

This paper introduces Sobolev training for optimization proxies, integrating solver sensitivities to improve approximation accuracy in both supervised and self-supervised settings, with significant empirical improvements on power flow and portfolio tasks.

Contribution

It develops a Sobolev training paradigm that incorporates derivative information into optimization proxies, enhancing their accuracy and reliability in large-scale optimization problems.

Findings

Supervised Sobolev training reduces mean squared error by up to 56%.

It decreases median worst case constraint violation by up to 400%.

Self supervised Sobolev training halves the optimality gap in certain portfolio tasks.

Abstract

Optimization proxies - machine learning models trained to approximate the solution mapping of parametric optimization problems in a single forward pass - offer dramatic reductions in inference time compared to traditional iterative solvers. This work investigates the integration of solver sensitivities into such end to end proxies via a Sobolev training paradigm and does so in two distinct settings: (i) fully supervised proxies, where exact solver outputs and sensitivities are available, and (ii) self supervised proxies that rely only on the objective and constraint structure of the underlying optimization problem. By augmenting the standard training loss with directional derivative information extracted from the solver, the proxy aligns both its predicted solutions and local derivatives with those of the optimizer. Under Lipschitz continuity assumptions on the true solution mapping, matching first order sensitivities is shown to yield uniform approximation error proportional to the training set covering radius. Empirically, different impacts are observed in each studied setting. On three large Alternating Current Optimal Power Flow benchmarks, supervised Sobolev training cuts mean squared error by up to 56 percent and the median worst case constraint violation by up to 400 percent while keeping the optimality gap below 0.22 percent. For a mean variance portfolio task trained without labeled solutions, self supervised Sobolev training halves the average optimality gap in the medium risk region (standard deviation above 10 percent of budget) and matches the baseline elsewhere. Together, these results highlight Sobolev training whether supervised or self supervised as a path to fast reliable surrogates for safety critical large scale optimization workloads.