Fundamental Tensor Operations for Large-Scale Data Analysis in Tensor Train Formats

TL;DR

This paper extends tensor operation definitions, explores low-rank tensor approximations like TT, and demonstrates their application in large-scale numerical analysis to overcome dimensionality challenges.

Contribution

It introduces effective tensor operations and low-rank approximation techniques, linking them for large-scale data analysis in tensor train formats.

Findings

Effective tensor operations for large-scale data

Low-rank tensor approximations like TT improve computational efficiency

Applications in solving large-scale optimization problems

Abstract

We discuss extended definitions of linear and multilinear operations such as Kronecker, Hadamard, and contracted products, and establish links between them for tensor calculus. Then we introduce effective low-rank tensor approximation techniques including Candecomp/Parafac (CP), Tucker, and tensor train (TT) decompositions with a number of mathematical and graphical representations. We also provide a brief review of mathematical properties of the TT decomposition as a low-rank approximation technique. With the aim of breaking the curse-of-dimensionality in large-scale numerical analysis, we describe basic operations on large-scale vectors, matrices, and high-order tensors represented by TT decomposition. The proposed representations can be used for describing numerical methods based on TT decomposition for solving large-scale optimization problems such as systems of linear equations and symmetric eigenvalue problems.