Linear-Scaling Tensor Train Sketching

TL;DR

This paper introduces the TTStack sketch, a new structured random projection for tensor train formats that achieves linear scaling in tensor order and subspace dimension, improving efficiency over previous methods.

Contribution

The paper proposes TTStack, a unified TT-adapted sketching operator with provable subspace embedding and injection properties, reducing exponential scaling issues in tensor train sketching.

Findings

TTStack satisfies OSE and OSI properties with linear dependence on tensor order

Provides quasi-optimal error bounds for tensor factorizations

Numerical experiments confirm theoretical advantages

Abstract

We introduce the TTStack sketch, a structured random projection tailored to the tensor train (TT) format that unifies existing TT-adapted sketching operators. By varying two integer parameters $P$ and $R$, TTStack interpolates between the Khatri-Rao sketch ($R=1$) and the Gaussian TT sketch ($P=1$). We prove that TTStack satisfies an oblivious subspace embedding (OSE) property with parameters $R = \mathcal{O}(d(r+\log 1/\delta))$ and $P = \mathcal{O}(\varepsilon^{-2})$, and an oblivious subspace injection (OSI) property under the condition $R = \mathcal{O}(d)$ and $P = \mathcal{O}(\varepsilon^{-2}(r + \log r/\delta))$. Both guarantees depend only linearly on the tensor order $d$ and on the subspace dimension $r$, in contrast to prior constructions that suffer from exponential scaling in $d$. As direct consequences, we derive quasi-optimal error bounds for the QB factorization and randomized TT rounding. The theoretical results are supported by numerical experiments on synthetic tensors, Hadamard products, and a quantum chemistry application.