Numerical Stability of the Nystr\"om Method

TL;DR

This paper investigates the numerical stability of the Nyström method, providing conditions and guidelines to ensure stable and accurate kernel approximations in large-scale computations.

Contribution

It establishes theoretical conditions for stability and offers practical strategies for stable implementation of the Nyström method.

Findings

Stability depends on the choice of column subsets.

Proper implementation of the pseudoinverse enhances stability.

Theoretical results are supported by experiments.

Abstract

The Nystr\"om method is a widely used technique for improving the scalability of kernel-based algorithms, including kernel ridge regression, spectral clustering, and Gaussian processes. Despite its popularity, the numerical stability of the method has remained largely an unresolved problem. In particular, the pseudo-inversion of the submatrix involved in the Nystr\"om method may pose stability issues as the submatrix is likely to be ill-conditioned, resulting in numerically poor approximation. In this work, we establish conditions under which the Nystr\"om method is numerically stable. We show that stability can be achieved through an appropriate choice of column subsets and a careful implementation of the pseudoinverse. Our results and experiments provide theoretical justification and practical guidance for the stable application of the Nystr\"om method in large-scale kernel computations.