Recursive Sketched Interpolation: Efficient Hadamard Products of Tensor Trains

TL;DR

This paper introduces Recursive Sketched Interpolation (RSI), an efficient algorithm for computing Hadamard products of tensor trains with reduced computational complexity, enabling scalable tensor operations in high-dimensional applications.

Contribution

The paper proposes RSI, a novel randomized sketching method that reduces the complexity of Hadamard tensor train products from (^4) to (^3), improving scalability.

Findings

RSI achieves (^3) complexity for Hadamard products.

Benchmarks show RSI outperforms traditional methods in scalability.

RSI maintains accuracy comparable to existing approaches.

Abstract

The Hadamard product of two tensors in the tensor-train (TT) format is a fundamental operation across various applications, such as TT-based function multiplication for nonlinear differential equations or convolutions. However, conventional methods for computing this product typically scale as at least $\mathcal{O}(\chi^4)$ with respect to the TT bond dimension (TT-rank) $\chi$, creating a severe computational bottleneck in practice. By combining randomized tensor-train sketching with slice selection via interpolative decomposition, we introduce Recursive Sketched Interpolation (RSI), a ``scale product'' algorithm that computes the Hadamard product of TTs at a computational cost of $\mathcal{O}(\chi^3)$. Benchmarks across various TT scenarios demonstrate that RSI offers superior scalability compared to traditional methods while maintaining comparable accuracy. We generalize RSI to compute more complex operations, including Hadamard products of multiple TTs and other element-wise nonlinear mappings, without increasing the complexity beyond $\mathcal{O}(\chi^3)$.