On Tensor Train Rank Minimization: Statistical Efficiency and Scalable Algorithm

TL;DR

This paper introduces a convex relaxation and a scalable algorithm for tensor train rank minimization, providing statistical guarantees and demonstrating effectiveness on real higher-order tensors.

Contribution

It presents the first convex relaxation for TT rank minimization with theoretical error bounds and an efficient randomized alternating optimization algorithm.

Findings

Theoretical error bounds for tensor completion using TT decomposition.

An efficient randomized optimization algorithm with linear time complexity.

Empirical validation on real higher-order tensor data.

Abstract

Tensor train (TT) decomposition provides a space-efficient representation for higher-order tensors. Despite its advantage, we face two crucial limitations when we apply the TT decomposition to machine learning problems: the lack of statistical theory and of scalable algorithms. In this paper, we address the limitations. First, we introduce a convex relaxation of the TT decomposition problem and derive its error bound for the tensor completion task. Next, we develop an alternating optimization method with a randomization technique, in which the time complexity is as efficient as the space complexity is. In experiments, we numerically confirm the derived bounds and empirically demonstrate the performance of our method with a real higher-order tensor.