Optimal High-order Tensor SVD via Tensor-Train Orthogonal Iteration

TL;DR

This paper introduces a new tensor-train orthogonal iteration algorithm for high-order tensor SVD, achieving minimax optimality and demonstrating effectiveness in tensor estimation, dimension reduction, and real-world data applications.

Contribution

The paper proposes the TTOI algorithm for efficient high-order tensor SVD with theoretical optimality guarantees and practical applications.

Findings

TTOI achieves minimax optimal estimation error.

The algorithm is computationally efficient and scalable.

Successful applications to real data and numerical simulations.

Abstract

This paper studies a general framework for high-order tensor SVD. We propose a new computationally efficient algorithm, tensor-train orthogonal iteration (TTOI), that aims to estimate the low tensor-train rank structure from the noisy high-order tensor observation. The proposed TTOI consists of initialization via TT-SVD (Oseledets, 2011) and new iterative backward/forward updates. We develop the general upper bound on estimation error for TTOI with the support of several new representation lemmas on tensor matricizations. By developing a matching information-theoretic lower bound, we also prove that TTOI achieves the minimax optimality under the spiked tensor model. The merits of the proposed TTOI are illustrated through applications to estimation and dimension reduction of high-order Markov processes, numerical studies, and a real data example on New York City taxi travel records. The software of the proposed algorithm is available online$^6$.