An Overview of GPU-based First-Order Methods for Linear Programming and Extensions

TL;DR

This survey reviews recent GPU-based first-order methods for linear programming, focusing on the design, enhancements, theoretical analysis, and extensions to other optimization problems, highlighting the efficiency and scalability of cuPDLP.

Contribution

It provides a comprehensive overview of GPU-accelerated first-order methods for LP, introduces the cuPDLP solver, and discusses theoretical and practical advancements including extensions beyond LP.

Findings

cuPDLP demonstrates superior scalability and efficiency compared to traditional solvers.

Adaptive restarts and preconditioning significantly improve convergence.

Theoretical framework covers sublinear and linear convergence under sharpness conditions.

Abstract

The rapid progress in GPU computing has revolutionized many fields, yet its potential in mathematical programming, such as linear programming (LP), has only recently begun to be realized. This survey aims to provide a comprehensive overview of recent advancements in GPU-based first-order methods for LP, with a particular focus on the design and development of cuPDLP. We begin by presenting the design principles and algorithmic foundation of the primal-dual hybrid gradient (PDHG) method, which forms the core of the solver. Practical enhancements, such as adaptive restarts, preconditioning, Halpern-type acceleration and infeasibility detection, are discussed in detail, along with empirical comparisons against industrial-grade solvers, highlighting the scalability and efficiency of cuPDLP. We also provide a unified theoretical framework for understanding PDHG, covering both classical and recent results on sublinear and linear convergence under sharpness conditions. Finally, we extend the discussion to GPU-based optimization beyond LP, including quadratic, semidefinite, conic, and nonlinear programming.