Sparse and Low-rank Tensor Estimation via Cubic Sketchings

TL;DR

This paper introduces a novel framework for estimating sparse and low-rank tensors from cubic sketchings, combining non-convex algorithms with theoretical guarantees for exact and stable recovery.

Contribution

It presents a new two-stage non-convex method for tensor estimation, with non-asymptotic analysis and rate-optimal guarantees, along with high-order concentration inequalities.

Findings

Exact recovery in noiseless case

Stable recovery with high probability in noisy case

Rate-optimal performance under certain conditions

Abstract

In this paper, we propose a general framework for sparse and low-rank tensor estimation from cubic sketchings. A two-stage non-convex implementation is developed based on sparse tensor decomposition and thresholded gradient descent, which ensures exact recovery in the noiseless case and stable recovery in the noisy case with high probability. The non-asymptotic analysis sheds light on an interplay between optimization error and statistical error. The proposed procedure is shown to be rate-optimal under certain conditions. As a technical by-product, novel high-order concentration inequalities are derived for studying high-moment sub-Gaussian tensors. An interesting tensor formulation illustrates the potential application to high-order interaction pursuit in high-dimensional linear regression.