Recent and Upcoming Developments in Randomized Numerical Linear Algebra for Machine Learning

TL;DR

This paper reviews recent advances and future challenges in Randomized Numerical Linear Algebra (RandNLA), emphasizing its growing importance in machine learning, data analysis, and the integration of new hardware and theoretical insights.

Contribution

It provides a comprehensive overview of RandNLA's recent developments, highlighting new theoretical and practical challenges driven by hardware trends and machine learning advances.

Findings

RandNLA has matured as a field with practical algorithms.

Recent hardware and machine learning trends pose new challenges.

Increased integration of RandNLA into core numerical libraries.

Abstract

Large matrices arise in many machine learning and data analysis applications, including as representations of datasets, graphs, model weights, and first and second-order derivatives. Randomized Numerical Linear Algebra (RandNLA) is an area which uses randomness to develop improved algorithms for ubiquitous matrix problems. The area has reached a certain level of maturity; but recent hardware trends, efforts to incorporate RandNLA algorithms into core numerical libraries, and advances in machine learning, statistics, and random matrix theory, have lead to new theoretical and practical challenges. This article provides a self-contained overview of RandNLA, in light of these developments.