Spectral Methods for Matrices and Tensors

TL;DR

This survey reviews recent advances in spectral methods, highlighting their applications to matrices and tensors in numerical and discrete optimization problems, emphasizing low-rank tensor approximations and randomized sampling techniques.

Contribution

It provides a comprehensive overview of spectral methods applied to matrices and tensors, including new algorithms for low-rank tensor approximation and sampling strategies for matrix estimation.

Findings

Sampling random submatrices estimates singular values effectively.

Spectral methods extend to tensor low-rank approximation.

Applications to Max-r-CSPs and numerical tensor problems.

Abstract

While Spectral Methods have long been used for Principal Component Analysis, this survey focusses on work over the last 15 years with three salient features: (i) Spectral methods are useful not only for numerical problems, but also discrete optimization problems (Constraint Optimization Problems - CSP's) like the max. cut problem and similar mathematical considerations underlie both areas. (ii) Spectral methods can be extended to tensors. The theory and algorithms for tensors are not as simple/clean as for matrices, but the survey describes methods for low-rank approximation which extend to tensors. These tensor approximations help us solve Max-$r$-CSP's for $r>2$ as well as numerical tensor problems. (iii) Sampling on the fly plays a prominent role in these methods. A primary result is that for any matrix, a random submatrix of rows/columns picked with probabilities proportional to the squared lengths (of rows/columns), yields estimates of the singular values as well as an approximation to the whole matrix.