Efficient error and variance estimation for randomized matrix computations

TL;DR

This paper introduces fast diagnostics for error and variance estimation in randomized matrix algorithms, enhancing their reliability and guiding parameter choices in scientific computing and machine learning.

Contribution

It proposes two new diagnostics: a leave-one-out error estimator and a jackknife variance estimator, tailored for randomized low-rank approximation algorithms.

Findings

Diagnostics are rapid to compute for randomized SVD and Nyström methods.

They provide valuable information for assessing output quality.

They help guide algorithmic parameter selection.

Abstract

Randomized matrix algorithms have become workhorse tools in scientific computing and machine learning. To use these algorithms safely in applications, they should be coupled with posterior error estimates to assess the quality of the output. To meet this need, this paper proposes two diagnostics: a leave-one-out error estimator for randomized low-rank approximations and a jackknife resampling method to estimate the variance of the output of a randomized matrix computation. Both of these diagnostics are rapid to compute for randomized low-rank approximation algorithms such as the randomized SVD and randomized Nystr\"om approximation, and they provide useful information that can be used to assess the quality of the computed output and guide algorithmic parameter choices.