High-Performance Out-of-core Block Randomized Singular Value Decomposition on GPU

TL;DR

This paper introduces a GPU-accelerated block randomized SVD algorithm capable of efficiently processing large-scale data that exceeds GPU memory, significantly speeding up computations in machine learning and computer vision tasks.

Contribution

It presents a novel out-of-core block randomized SVD algorithm that fully utilizes GPU architecture for large-scale data processing, outperforming existing methods in speed while maintaining accuracy.

Findings

The proposed BRSVD outperforms existing randomized SVD methods in speed.

It effectively processes data larger than GPU memory.

Application to robust PCA shows significant speedup in vision tasks.

Abstract

Fast computation of singular value decomposition (SVD) is of great interest in various machine learning tasks. Recently, SVD methods based on randomized linear algebra have shown significant speedup in this regime. This paper attempts to further accelerate the computation by harnessing a modern computing architecture, namely graphics processing unit (GPU), with the goal of processing large-scale data that may not fit in the GPU memory. It leads to a new block randomized algorithm that fully utilizes the power of GPUs and efficiently processes large-scale data in an out-of- core fashion. Our experiment shows that the proposed block randomized SVD (BRSVD) method outperforms existing randomized SVD methods in terms of speed with retaining the same accuracy. We also show its application to convex robust principal component analysis, which shows significant speedup in computer vision applications.