Introduction to Tensor Decompositions and their Applications in Machine Learning

TL;DR

This paper provides a comprehensive overview of tensor concepts, decompositions, algorithms, and their applications in machine learning, highlighting their advantages, challenges, and current research landscape.

Contribution

It offers a broad, accessible introduction to tensor decompositions, including algorithms, applications, and a case study, filling a gap in machine learning literature.

Findings

Tensors offer unique decomposition properties beneficial for unsupervised learning.

Tensor algorithms can effectively model multi-relational and temporal data.

Software libraries facilitate practical tensor applications in machine learning.

Abstract

Tensors are multidimensional arrays of numerical values and therefore generalize matrices to multiple dimensions. While tensors first emerged in the psychometrics community in the $20^{\text{th}}$ century, they have since then spread to numerous other disciplines, including machine learning. Tensors and their decompositions are especially beneficial in unsupervised learning settings, but are gaining popularity in other sub-disciplines like temporal and multi-relational data analysis, too. The scope of this paper is to give a broad overview of tensors, their decompositions, and how they are used in machine learning. As part of this, we are going to introduce basic tensor concepts, discuss why tensors can be considered more rigid than matrices with respect to the uniqueness of their decomposition, explain the most important factorization algorithms and their properties, provide concrete examples of tensor decomposition applications in machine learning, conduct a case study on tensor-based estimation of mixture models, talk about the current state of research, and provide references to available software libraries.