Differentiation Through Black-Box Quadratic Programming Solvers

TL;DR

This paper introduces dQP, a flexible, solver-agnostic framework for differentiating quadratic programming solutions, enabling broader integration in neural networks and optimization tasks without solver restrictions.

Contribution

The authors present dQP, a modular framework that allows plug-and-play differentiation of any QP solver, decoupling solution computation from differentiation for enhanced flexibility.

Findings

Supports over 15 state-of-the-art solvers

Demonstrates robustness and scalability in large-scale sparse problems

Achieves minimal overhead in differentiation process

Abstract

Differentiable optimization has attracted significant research interest, particularly for quadratic programming (QP). Existing approaches for differentiating the solution of a QP with respect to its defining parameters often rely on specific integrated solvers. This integration limits their applicability, including their use in neural network architectures and bi-level optimization tasks, restricting users to a narrow selection of solver choices. To address this limitation, we introduce dQP, a modular and solver-agnostic framework for plug-and-play differentiation of virtually any QP solver. A key insight we leverage to achieve modularity is that, once the active set of inequality constraints is known, both the solution and its derivative can be expressed using simplified linear systems that share the same matrix. This formulation fully decouples the computation of the QP solution from its differentiation. Building on this result, we provide a minimal-overhead, open-source implementation ( https://github.com/cwmagoon/dQP ) that seamlessly integrates with over 15 state-of-the-art solvers. Comprehensive benchmark experiments demonstrate dQP's robustness and scalability, particularly highlighting its advantages in large-scale sparse problems.