ISLET: Fast and Optimal Low-rank Tensor Regression via Importance Sketching

TL;DR

ISLET introduces a fast, optimal low-rank tensor regression method using importance sketching, achieving minimax optimality and superior computational efficiency for large-scale tensor data.

Contribution

The paper proposes a novel importance sketching approach for low-rank tensor regression that is both theoretically optimal and computationally efficient, especially for large tensors.

Findings

Achieves minimax optimal mean-squared error under low-rank Tucker assumptions.

Performs reliably on tensors with dimensions up to 10^8.

Significantly faster and more storage-efficient than existing methods.

Abstract

In this paper, we develop a novel procedure for low-rank tensor regression, namely \emph{\underline{I}mportance \underline{S}ketching \underline{L}ow-rank \underline{E}stimation for \underline{T}ensors} (ISLET). The central idea behind ISLET is \emph{importance sketching}, i.e., carefully designed sketches based on both the responses and low-dimensional structure of the parameter of interest. We show that the proposed method is sharply minimax optimal in terms of the mean-squared error under low-rank Tucker assumptions and under randomized Gaussian ensemble design. In addition, if a tensor is low-rank with group sparsity, our procedure also achieves minimax optimality. Further, we show through numerical study that ISLET achieves comparable or better mean-squared error performance to existing state-of-the-art methods while having substantial storage and run-time advantages including capabilities for parallel and distributed computing. In particular, our procedure performs reliable estimation with tensors of dimension $p = O(10^8)$ and is $1$ or $2$ orders of magnitude faster than baseline methods.