Accelerating Low-Rank Factorization-Based Semidefinite Programming Algorithms on GPU

TL;DR

This paper introduces GPU-accelerated techniques for low-rank factorization-based semidefinite programming, enabling the solution of extremely large-scale problems with unprecedented speed and scalability.

Contribution

It presents novel GPU acceleration methods for low-rank SDP solvers, significantly improving efficiency and scalability over previous CPU-based approaches.

Findings

Solved MaxCut problems with 10^7 x 10^7 matrices in 10 seconds to 1 minute.

Handled 170 million x 170 million MaxCut problems in minutes.

Solved 20 million x 20 million matrix completion problems with 200 million constraints in minutes.

Abstract

In this paper, we address a long-standing challenge: how to achieve both efficiency and scalability in solving semidefinite programming problems. We propose breakthrough acceleration techniques for a wide range of low-rank factorization-based first-order methods using GPUs, making the computation much more efficient and scalable. To illustrate the idea and effectiveness of our approach, we use the low-rank factorization-based SDP solver, LoRADS, as an example, which involves both the classic Burer-Monterio method and a novel splitting scheme with a starting logarithmic rank. Our numerical results demonstrate that the accelerated GPU version of LoRADS, cuLoRADS, can solve huge-scale semidefinite programming problems with remarkable efficiency. By effectively leveraging GPU computational power, cuLoRADS exhibits outstanding performance. Specifically, it can solve a set of MaxCut problems with $10^7 \times 10^7$ matrix variables in 10 seconds to 1 minute each on an NVIDIA H100 GPU with 80GB memory, whereas previous solvers demonstrated the capability of handling problems of this scale, required at least dozens of hours per problem on CPUs. Additionally, cuLoRADS shows exceptional scalability by solving 1) a MaxCut problem with a $170 \text{ million} \times 170 \text{ million}$ matrix variable and 2) a Matrix Completion problem with a 20 million $\times$ 20 million matrix variable and approximately 200 million constraints, both in a matter of minutes.