TT-LSQR For Tensor Least Squares Problems and Application to Data Mining *

TL;DR

This paper introduces TT-LSQR, a tensor-train based extension of the LSQR algorithm for tensor least squares problems, with applications to data mining and document classification.

Contribution

It develops a tensor-train implementation of LSQR with sketching techniques to efficiently solve large tensor least squares problems.

Findings

Tensor-train LSQR effectively solves high-dimensional tensor least squares problems.

Sketching reduces computational costs significantly.

Application demonstrated in document classification tasks.

Abstract

We are interested in the numerical solution of the tensor least squares problem \[ \min_{\mathcal{X}} \| \mathcal{F} - \sum_{i =1}^{\ell} \mathcal{X} \times_1 A_1^{(i)} \times_2 A_2^{(i)} \cdots \times_d A_d^{(i)} \|_F, \] where $\mathcal{X}\in\mathbb{R}^{m_1 \times m_2 \times \cdots \times m_d}$, $\mathcal{F}\in\mathbb{R}^{n_1\times n_2 \times \cdots \times n_d}$ are tensors with $d$ dimensions, and the coefficients $A_j^{(i)}$ are tall matrices of conforming dimensions. We first describe a tensor implementation of the classical LSQR method by Paige and Saunders, using the tensor-train representation as key ingredient. We also show how to incorporate sketching to lower the computational cost of dealing with the tall matrices $A_j^{(i)}$. We then use this methodology to address a problem in information retrieval, the classification of a new query document among already categorized documents, according to given keywords.