Projected Tensor-Tensor Products for Efficient Computation of Optimal Multiway Data Representations

TL;DR

This paper introduces a projected tensor-tensor product that reduces computational costs in multiway data representations while maintaining key linear algebra properties, enabling efficient large-scale tensor decompositions.

Contribution

It proposes a novel projected tensor-tensor product that relaxes invertibility constraints, significantly lowering computational complexity and improving tensor factorization efficiency.

Findings

Reduces computational complexity by an order of magnitude.

Maintains matrix mimetic properties and optimality of representations.

Outperforms non-matrix-mimetic tensor factorizations in experiments.

Abstract

Tensor decompositions have become essential tools for feature extraction and compression of multiway data. Recent advances in tensor operators have enabled desirable properties of standard matrix algebra to be retained for multilinear factorizations. Behind this matrix-mimetic tensor operation is an invertible matrix whose size depends quadratically on certain dimensions of the data. As a result, for large-scale multiway data, the invertible matrix can be computationally demanding to apply and invert and can lead to inefficient tensor representations in terms of construction and storage costs. In this work, we propose a new projected tensor-tensor product that relaxes the invertibility restriction to reduce computational overhead and still preserves fundamental linear algebraic properties. The transformation behind the projected product is a tall-and-skinny matrix with unitary columns, which depends only linearly on certain dimensions of the data, thereby reducing computational complexity by an order of magnitude. We provide extensive theory to prove the matrix mimeticity and the optimality of compressed representations within the projected product framework. We further prove that projected-product-based approximations outperform a comparable, non-matrix-mimetic tensor factorization. We support the theoretical findings and demonstrate the practical benefits of projected products through numerical experiments on video and hyperspectral imaging data.