cuHALLaR: A GPU Accelerated Low-Rank Augmented Lagrangian Method for Large-Scale Semidefinite Programming

TL;DR

cuHALLaR is a GPU-accelerated implementation of a low-rank method for large-scale semidefinite programming, achieving significant speedups and enabling the solution of massive problems efficiently.

Contribution

This paper presents cuHALLaR, a Julia-based GPU implementation of the HALLaR method, optimized for large-scale SDP problems, demonstrating substantial performance improvements over existing solutions.

Findings

Speedups of 30-140x over CPU implementations.

Ability to solve problems with up to 8 million dimensions in 142 seconds.

Effective handling of massive SDP problems with high precision.

Abstract

This paper introduces cuHALLaR, a GPU-accelerated implementation of the HALLaR method proposed in Monteiro et al. 2024 for solving large-scale semidefinite programming (SDP) problems. We demonstrate how our Julia-based implementation efficiently uses GPU parallelism through optimization of simple, but key, operations, including linear maps, adjoints, and gradient evaluations. Extensive numerical experiments across three SDP problem classes, i.e., maximum stable set, matrix completion, and phase retrieval show significant performance improvements over both CPU implementations and existing GPU-based solvers. For the largest instances, cuHALLaR achieves speedups of 30-140x on matrix completion problems, up to 135x on maximum stable set problems for Hamming graphs with 8.4 million vertices, and 15-47x on phase retrieval problems with dimensions up to 3.2 million. Our approach efficiently handles massive problems with dimensions up to (n,m) equal to (8 million, 300 million) with high precision, solving matrix completion instances with over 8 million rows and columns in just 142 seconds. These results establish cuHALLaR as a very promising GPU-based method for solving large-scale semidefinite programs.