Column and row subset selection using nuclear scores: algorithms and theory for Nystr\"{o}m approximation, CUR decomposition, and graph Laplacian reduction

TL;DR

This paper introduces unified, efficient algorithms for column subset selection that are theoretically guaranteed and applicable to various low-rank approximation tasks, including CUR decomposition and graph Laplacian reduction.

Contribution

It develops deterministic and randomized algorithms with performance guarantees for column selection, extending the theory and application of Nyström approximation and CUR decomposition.

Findings

Algorithms outperform existing methods in real-world tasks.

Theoretical bounds compare favorably to DPP sampling.

Effective for kernel approximation and graph Laplacian functions.

Abstract

Column selection is an essential tool for structure-preserving low-rank approximation, with wide-ranging applications across many fields, such as data science, machine learning, and theoretical chemistry. In this work, we develop unified methodologies for fast, efficient, and theoretically guaranteed column selection. First we derive and implement a sparsity-exploiting deterministic algorithm applicable to tasks including kernel approximation and CUR decomposition. Next, we develop a matrix-free formalism relying on a randomization scheme satisfying guaranteed concentration bounds, applying this construction both to CUR decomposition and to the approximation of matrix functions of graph Laplacians. Importantly, the randomization is only relevant for the computation of the scores that we use for column selection, not the selection itself given these scores. For both deterministic and matrix-free algorithms, we bound the performance favorably relative to the expected performance of determinantal point process (DPP) sampling and, in select scenarios, that of exactly optimal subset selection. The general case requires new analysis of the DPP expectation. Finally, we demonstrate strong real-world performance of our algorithms on a diverse set of example approximation tasks.