Improving CUR Matrix Decomposition and the Nystr\"{o}m Approximation via Adaptive Sampling

TL;DR

This paper introduces improved CUR and Nyström matrix approximation algorithms using adaptive sampling, providing tighter error bounds, efficiency, and theoretical analysis without special data assumptions.

Contribution

It establishes a general error bound for adaptive sampling and proposes more accurate, efficient CUR and Nyström algorithms with theoretical guarantees.

Findings

New error bounds for adaptive sampling algorithms

More accurate CUR and Nyström algorithms with expected relative-error bounds

Theoretical analysis of lower error bounds for Nyström methods

Abstract

The CUR matrix decomposition and the Nystr\"{o}m approximation are two important low-rank matrix approximation techniques. The Nystr\"{o}m method approximates a symmetric positive semidefinite matrix in terms of a small number of its columns, while CUR approximates an arbitrary data matrix by a small number of its columns and rows. Thus, CUR decomposition can be regarded as an extension of the Nystr\"{o}m approximation. In this paper we establish a more general error bound for the adaptive column/row sampling algorithm, based on which we propose more accurate CUR and Nystr\"{o}m algorithms with expected relative-error bounds. The proposed CUR and Nystr\"{o}m algorithms also have low time complexity and can avoid maintaining the whole data matrix in RAM. In addition, we give theoretical analysis for the lower error bounds of the standard Nystr\"{o}m method and the ensemble Nystr\"{o}m method. The main theoretical results established in this paper are novel, and our analysis makes no special assumption on the data matrices.