Solving Linear Systems on a GPU with Hierarchically Off-Diagonal Low-Rank Approximations

TL;DR

This paper develops GPU algorithms for efficiently solving large linear systems with hierarchically off-diagonal low-rank matrices, enabling fast direct solutions or preconditioning for applications in machine learning and physics.

Contribution

It introduces scalable GPU algorithms for HODLR matrix factorization and application, with tunable accuracy for direct solving or preconditioning.

Findings

Solved large problems with millions of unknowns in seconds on a GPU.

Algorithms scale nearly linearly with matrix size for fixed ranks.

Flexible accuracy allows for high-precision or robust preconditioning.

Abstract

We are interested in solving linear systems arising from three applications: (1) kernel methods in machine learning, (2) discretization of boundary integral equations from mathematical physics, and (3) Schur complements formed in the factorization of many large sparse matrices. The coefficient matrices are often data-sparse in the sense that their off-diagonal blocks have low numerical ranks; specifically, we focus on "hierarchically off-diagonal low-rank (HODLR)" matrices. We introduce algorithms for factorizing HODLR matrices and for applying the factorizations on a GPU. The algorithms leverage the efficiency of batched dense linear algebra, and they scale nearly linearly with the matrix size when the numerical ranks are fixed. The accuracy of the HODLR-matrix approximation is a tunable parameter, so we can construct high-accuracy fast direct solvers or low-accuracy robust preconditioners. Numerical results show that we can solve problems with several millions of unknowns in a couple of seconds on a single GPU.