Fast elementwise operations on tensor trains with alternating cross interpolation

TL;DR

This paper introduces the alternating cross interpolation (ACI) algorithm that accelerates elementwise operations on tensor trains from $O(mbda^4)$ to $O(mbda^3)$, maintaining error control and improving efficiency.

Contribution

The paper presents a novel ACI algorithm that reduces the computational complexity of elementwise tensor train operations, enabling faster calculations in practical applications.

Findings

ACI achieves $O(mbda^3)$ complexity for elementwise TT operations.

The algorithm maintains error control during computations.

Benchmarks show significant speedup for common TT ranks.

Abstract

Tensor trains (TTs), also known as matrix product states (MPS), are compressed representations of high-dimensional data that can be efficiently manipulated to perform calculations on the data. In many applications, such as TT-based solvers for nonlinear partial differential equations, the most expensive step is an elementwise multiplication or similar elementwise operation on multiple TTs. Known error-controlled algorithms for such operations scale as $O(\chi^4)$, where $\chi$ is the TT rank. If the rank of the output is smaller than $\chi^2$, it is possible to formulate algorithms with better scaling. In this work, we present the alternating cross interpolation (ACI) algorithm that performs such operations in $O(\chi^3)$, while maintaining error control. We demonstrate these properties on benchmark problems, achieving a significant speedup for TT ranks that are commonly encountered in practical applications.