Improved Analysis of Khatri-Rao Random Projections and Applications

TL;DR

This paper improves the theoretical understanding of Khatri-Rao random projections (KRPs) in large-scale matrix and tensor decompositions, demonstrating their efficiency and accuracy through new algorithms and analysis.

Contribution

The authors provide an enhanced theoretical analysis of KRPs, develop a new low-rank approximation algorithm for block-structured matrices, and accelerate tensor computations with proven guarantees.

Findings

KRPs offer a computationally cheaper alternative to Gaussian random matrices.

The new algorithms achieve accurate low-rank approximations in practice.

Numerical experiments confirm the efficiency and effectiveness of the proposed methods.

Abstract

Randomization has emerged as a powerful set of tools for large-scale matrix and tensor decompositions. Randomized algorithms involve computing sketches with random matrices. A prevalent approach is to take the random matrix as a standard Gaussian random matrix, for which the theory is well developed. However, this approach has the drawback that the cost of generating and multiplying by the random matrix can be prohibitively expensive. Khatri-Rao random projections (KRPs), obtained by sketching with Khatri-Rao products of random matrices, offer a viable alternative and are much cheaper to generate. However, the theoretical guarantees of using KRPs are much more pessimistic compared to their accuracy observed in practice. We attempt to close this gap by obtaining improved analysis of the use of KRPs in matrix and tensor low-rank decompositions. We propose and analyze a new algorithm for low-rank approximations of block-structured matrices (e.g., block Hankel) using KRPs. We also show how to accelerate tensor computations in the Tucker format using KRPs and give theoretical guarantees of the resulting low-rank approximations. Numerical experiments on synthetic and real-world tensors show the computational benefits of the proposed methods.