A New Sampling Technique for Tensors

TL;DR

This paper introduces a biased random sampling method for third-order tensors that reduces the number of elements needed for tensor approximation tasks, significantly speeding up tensor algorithms in machine learning.

Contribution

The paper presents a novel sampling technique that efficiently selects a small subset of tensor elements to enable faster tensor approximation, completion, and factorization.

Findings

Reduces sampling complexity to O(n^{1.5}/ε^2) elements.

Achieves spectral approximation, tensor completion, and factorization with fewer samples.

Speeds up existing tensor algorithms by removing computational bottlenecks.

Abstract

In this paper we propose new techniques to sample arbitrary third-order tensors, with an objective of speeding up tensor algorithms that have recently gained popularity in machine learning. Our main contribution is a new way to select, in a biased random way, only $O(n^{1.5}/\epsilon^2)$ of the possible $n^3$ elements while still achieving each of the three goals: \\ {\em (a) tensor sparsification}: for a tensor that has to be formed from arbitrary samples, compute very few elements to get a good spectral approximation, and for arbitrary orthogonal tensors {\em (b) tensor completion:} recover an exactly low-rank tensor from a small number of samples via alternating least squares, or {\em (c) tensor factorization:} approximating factors of a low-rank tensor corrupted by noise. \\ Our sampling can be used along with existing tensor-based algorithms to speed them up, removing the computational bottleneck in these methods.