Sampling-Based Decomposition Algorithms for Arbitrary Tensor Networks

TL;DR

This paper introduces a sampling-based ALS algorithm for tensor network decomposition that leverages exact leverage score sampling to achieve sublinear per-iteration costs, demonstrated through feature extraction experiments.

Contribution

It presents a novel sampling framework for tensor network decomposition using leverage scores, enabling efficient ALS algorithms with theoretical and practical advantages.

Findings

Achieved sublinear per-iteration cost in tensor decomposition algorithms.

Demonstrated improved efficiency in feature extraction tasks.

Validated the approach against existing tensor decomposition methods.

Abstract

We show how to develop sampling-based alternating least squares (ALS) algorithms for decomposition of tensors into any tensor network (TN) format. Provided the TN format satisfies certain mild assumptions, resulting algorithms will have input sublinear per-iteration cost. Unlike most previous works on sampling-based ALS methods for tensor decomposition, the sampling in our framework is done according to the exact leverage score distribution of the design matrices in the ALS subproblems. We implement and test two tensor decomposition algorithms that use our sampling framework in a feature extraction experiment where we compare them against a number of other decomposition algorithms.