Condensed-space methods for nonlinear programming on GPUs

TL;DR

This paper presents two condensed-space interior-point methods optimized for GPU acceleration to efficiently solve large-scale nonlinear programming problems, demonstrating significant speedups over CPU implementations.

Contribution

It introduces and compares two novel condensed-space interior-point methods, HyKKT and LiftedKKT, implemented on GPUs for improved large-scale nonlinear programming performance.

Findings

GPUs achieve up to tenfold speedup over CPUs.

Condensed KKT systems are more ill-conditioned but still effective.

The methods perform well on benchmark problems.

Abstract

This paper explores two condensed-space interior-point methods to efficiently solve large-scale nonlinear programs on graphics processing units (GPUs). The interior-point method solves a sequence of symmetric indefinite linear systems, or Karush-Kuhn-Tucker (KKT) systems, which become increasingly ill-conditioned as we approach the solution. Solving a KKT system with traditional sparse factorization methods involve numerical pivoting, making parallelization difficult. A solution is to condense the KKT system into a symmetric positive-definite matrix and solve it with a Cholesky factorization, stable without pivoting. Although condensed KKT systems are more prone to ill-conditioning than the original ones, they exhibit structured ill-conditioning that mitigates the loss of accuracy. This paper compares the benefits of two recent condensed-space interior-point methods, HyKKT and LiftedKKT. We implement the two methods on GPUs using MadNLP.jl, an optimization solver interfaced with the NVIDIA sparse linear solver cuDSS and with the GPU-accelerated modeler ExaModels.jl. Our experiments on the PGLIB and the COPS benchmarks reveal that GPUs can attain up to a tenfold speed increase compared to CPUs when solving large-scale instances.