Low-Rank Tensor Networks for Dimensionality Reduction and Large-Scale Optimization Problems: Perspectives and Challenges PART 1

TL;DR

This paper reviews tensor network methods, especially Tucker and Tensor Train decompositions, as powerful tools for large-scale data analysis, dimensionality reduction, and optimization in machine learning and data mining.

Contribution

It provides a comprehensive overview of tensor network representations, their mathematical foundations, and their applications to large-scale data analysis and optimization problems.

Findings

Tensor networks enable efficient representation of multiway data.

Tensor decompositions facilitate dimensionality reduction and large-scale optimization.

The paper discusses extensions and generalizations of Tucker and TT decompositions.

Abstract

Machine learning and data mining algorithms are becoming increasingly important in analyzing large volume, multi-relational and multi--modal datasets, which are often conveniently represented as multiway arrays or tensors. It is therefore timely and valuable for the multidisciplinary research community to review tensor decompositions and tensor networks as emerging tools for large-scale data analysis and data mining. We provide the mathematical and graphical representations and interpretation of tensor networks, with the main focus on the Tucker and Tensor Train (TT) decompositions and their extensions or generalizations. Keywords: Tensor networks, Function-related tensors, CP decomposition, Tucker models, tensor train (TT) decompositions, matrix product states (MPS), matrix product operators (MPO), basic tensor operations, multiway component analysis, multilinear blind source separation, tensor completion, linear/multilinear dimensionality reduction, large-scale optimization problems, symmetric eigenvalue decomposition (EVD), PCA/SVD, huge systems of linear equations, pseudo-inverse of very large matrices, Lasso and Canonical Correlation Analysis (CCA) (This is Part 1)