Tensor Ranks for the Pedestrian for Dimension Reduction and Disentangling Interactions

TL;DR

This paper explores tensor ranks for dimension reduction and disentangling interactions in multivariate data, presenting algorithms for low-rank approximation and demonstrating their effectiveness on discretized functions.

Contribution

It introduces a systematic tensor algebra approach for low-rank tensor approximation and applies it to multivariate functions discretized on Cartesian grids.

Findings

Low-rank tensors effectively represent multivariate interactions.

The proposed algorithms efficiently compute best low-rank approximations.

Tensor ranks are generally low for discretized multivariate functions.

Abstract

A tensor is a multi-way array that can represent, in addition to a data set, the expression of a joint law or a multivariate function. As such it contains the description of the interactions between the variables corresponding to each of the entries. The rank of a tensor extends to arrays with more than two entries the notion of rank of a matrix, bearing in mind that there are several approaches to build such an extension. When the rank is one, the variables are separated, and when it is low, the variables are weakly coupled. Many calculations are simpler on tensors of low rank. Furthermore, approximating a given tensor by a low-rank tensor makes it possible to compute some characteristics of a table, such as the partition function when it is a joint law. In this note, we present in detail an integrated and progressive approach to approximate a given tensor by a tensor of lower rank, through a systematic use of tensor algebra. The notion of tensor is rigorously defined, then elementary but useful operations on tensors are presented. After recalling several different notions for extending the rank to tensors, we show how these elementary operations can be combined to build best low rank approximation algorithms. The last chapter is devoted to applying this approach to tensors constructed as the discretisation of a multivariate function, to show that on a Cartesian grid, the rank of such tensors is expected to be low.