GPU-Parallelizable Randomized Sketch-and-Precondition for Linear Regression using Sparse Sign Sketches

TL;DR

This paper demonstrates that sketch-and-precondition methods using sparse sign sketches are highly effective on GPU systems for large linear regression problems, offering a scalable and adaptable approach.

Contribution

It introduces a novel, parallelizable rejection-sampling method for generating sparse sign sketches optimized for GPU architectures.

Findings

Sketch-and-precondition with sparse sign sketches performs well on GPUs.

The proposed method is easily adaptable to various computing environments.

Numerical experiments show the approach's suitability for black-box least-squares solvers.

Abstract

A litany of theoretical and numerical results have established the sketch-and-precondition paradigm as a powerful approach to solving large linear regression problems in standard computing environments. Perhaps surprisingly, much less work has been done on understanding how sketch-and-precondition performs on graphics processing unit (GPU) systems. We address this gap by benchmarking an implementation of sketch-and-precondition based on sparse sign-sketches on single and multi-GPU systems. In doing so, we describe a novel, easily parallelized, rejection-sampling based method for generating sparse sign sketches. Our approach, which is particularly well-suited for GPUs, is easily adapted to a variety of computing environments. Taken as a whole, our numerical experiments indicate that sketch-and-precondition with sparse sign sketches is particularly well-suited for GPUs, and may be suitable for use in black-box least-squares solvers.