On the randomized SVD in infinite dimensions

TL;DR

This paper introduces a new infinite-dimensional randomized SVD method that avoids the pitfalls of prior approaches, providing error bounds comparable to finite-dimensional cases and connecting discretized approximations to the infinite-dimensional limit.

Contribution

A novel infinite-dimensional randomized SVD that does not depend on covariance operator choices, with error bounds matching finite-dimensional results and a new Nyström extension for trace class operators.

Findings

The new method achieves error bounds similar to finite-dimensional randomized SVD.

Discretized versions of the classical randomized SVD converge to the infinite-dimensional extension.

A novel Nyström approximation extension for positive semi-definite trace class operators is proposed.

Abstract

Randomized methods, such as the randomized SVD (singular value decomposition) and Nystr\"om approximation, are an effective way to compute low-rank approximations of large matrices. Motivated by applications to operator learning, Boull\'e and Townsend (FoCM, 2023) recently proposed an infinite-dimensional extension of the randomized SVD for a Hilbert-Schmidt operator $A$ that invokes randomness through a Gaussian process with a covariance operator $K$. While the non-isotropy introduced by $K$ allows one to incorporate prior information on $A$, an unfortunate choice may lead to unfavorable performance and large constants in the error bounds. In this work, we introduce a novel infinite-dimensional extension of the randomized SVD that does not require such a choice and enjoys error bounds that match those for the finite-dimensional case. Our extension implicitly uses isotropic random vectors, reflecting a choice commonly made in the finite-dimensional case. In fact, the theoretical results of this work show how the usual randomized SVD applied to a discretization of $A$ approaches our infinite-dimensional extension as the discretization gets refined, both in terms of error bounds and the Wasserstein distance. We also present and analyze a novel extension of the Nystr\"om approximation for self-adjoint positive semi-definite trace class operators.