Optimal Matrix-Mimetic Tensor Algebras via Variable Projection

TL;DR

This paper introduces a novel framework that jointly learns optimal linear transformations and tensor representations for multiway data, improving performance without prior data knowledge, using variable projection and Riemannian optimization.

Contribution

It proposes a new method combining variable projection and Riemannian optimization to learn invertible linear mappings and tensor representations simultaneously, enhancing matrix-mimetic tensor frameworks.

Findings

Framework effectively learns transformations without prior data info

Demonstrates improved results in applications like image compression

Provides theoretical guarantees on uniqueness and convergence

Abstract

Recent advances in {matrix-mimetic} tensor frameworks have made it possible to preserve linear algebraic properties for multilinear data analysis and, as a result, to obtain optimal representations of multiway data. Matrix mimeticity arises from interpreting tensors as operators that can be multiplied, factorized, and analyzed analogous to matrices. Underlying the tensor operation is an algebraic framework parameterized by an invertible linear transformation. The choice of linear mapping is crucial to representation quality and, in practice, is made heuristically based on expected correlations in the data. However, in many cases, these correlations are unknown and common heuristics lead to suboptimal performance. In this work, we simultaneously learn optimal linear mappings and corresponding tensor representations without relying on prior knowledge of the data. Our new framework explicitly captures the coupling between the transformation and representation using variable projection. We preserve the invertibility of the linear mapping by learning orthogonal transformations with Riemannian optimization. We provide original theory of uniqueness of the transformation and convergence analysis of our variable-projection-based algorithm. We demonstrate the generality of our framework through numerical experiments on a wide range of applications, including financial index tracking, image compression, and reduced order modeling. We have published all the code related to this work at https://github.com/elizabethnewman/star-M-opt.