Giga-scale Kernel Matrix Vector Multiplication on GPU

TL;DR

This paper introduces threem, a novel approximation method for kernel matrix-vector multiplication that achieves near-linear time and memory complexity, enabling large-scale computations on GPUs with significant speedups.

Contribution

The paper presents threem, a new approximation technique for KMVM that scales linearly in time and memory, suitable for very large datasets on GPU hardware.

Findings

threem achieves linear time and memory complexity with low error.

It can compute billion-point KMVMs in under a minute on high-end GPUs.

It improves existing GPU-based solvers' speed by 1.5-5.5 times with minimal accuracy loss.

Abstract

Kernel matrix-vector multiplication (KMVM) is a foundational operation in machine learning and scientific computing. However, as KMVM tends to scale quadratically in both memory and time, applications are often limited by these computational constraints. In this paper, we propose a novel approximation procedure coined \textit{Faster-Fast and Free Memory Method} ($\fthreem$) to address these scaling issues of KMVM for tall~($10^8\sim 10^9$) and skinny~($D\leq7$) data. Extensive experiments demonstrate that $\fthreem$ has empirical \emph{linear time and memory} complexity with a relative error of order $10^{-3}$ and can compute a full KMVM for a billion points \emph{in under a minute} on a high-end GPU, leading to a significant speed-up in comparison to existing CPU methods. We demonstrate the utility of our procedure by applying it as a drop-in for the state-of-the-art GPU-based linear solver FALKON, \emph{improving speed 1.5-5.5 times} at the cost of $<1\%$ drop in accuracy. We further demonstrate competitive results on \emph{Gaussian Process regression} coupled with significant speedups on a variety of real-world datasets.