A Differentiable Interior-Point Method in Single Precision

TL;DR

This paper presents a novel differentiable interior-point method optimized for single-precision arithmetic, enabling reliable solving and differentiation of constrained optimization problems in low-precision settings.

Contribution

It introduces an alternative complementarity representation that maintains spectral bounds in linear systems, improving robustness in single precision.

Findings

Enables interior-point methods to operate reliably in single precision.

Maintains spectral bounds near the solution, preventing ill-conditioning.

Demonstrates effectiveness in bilevel and end-to-end learning applications.

Abstract

Primal-dual interior-point methods solve constrained convex optimization problems to tight tolerances with speed and robustness. Their solutions are also efficiently differentiable with respect to the problem data through the implicit function theorem. However, the standard treatment of primal-dual complementarity makes the underlying linear systems increasingly ill-conditioned near the solution. While this ill-conditioning is often benign in double precision, it can be catastrophic in single precision, preventing interior-point methods from fully exploiting the accelerated hardware that underpins modern machine learning. This paper introduces a differentiable interior-point method designed for low-precision arithmetic. By using an alternative complementarity representation, we ensure that the underlying linear systems remain spectrally bounded -- even near the solution -- a property that is essential for computing accurate gradients and avoiding arithmetic exceptions. As a result, our method enables interior-point solvers to reliably solve and differentiate optimization problems in single precision that were previously confined to double precision. We demonstrate the approach through an ablation study against the standard interior-point formulation and applications in bilevel and end-to-end learning settings where differentiating through constrained optimization is essential. The source code is available at https://github.com/qpax-solver/qpax.