A Block-Randomized Stochastic Method with Importance Sampling for CP Tensor Decomposition

TL;DR

This paper introduces a novel stochastic gradient descent method with importance sampling for CP tensor decomposition, improving efficiency and performance by avoiding explicit KRP formation and providing theoretical guarantees.

Contribution

It proposes a new mini-batch stochastic gradient method with importance sampling for CP tensor decomposition, including adaptive step size and two sampling strategies, with theoretical analysis and superior empirical results.

Findings

Outperforms existing methods on synthetic data.

Effective importance sampling strategies improve convergence.

Theoretical analysis confirms convergence properties.

Abstract

One popular way to compute the CANDECOMP/PARAFAC (CP) decomposition of a tensor is to transform the problem into a sequence of overdetermined least squares subproblems with Khatri-Rao product (KRP) structure involving factor matrices. In this work, based on choosing the factor matrix randomly, we propose a mini-batch stochastic gradient descent method with importance sampling for those special least squares subproblems. Two different sampling strategies are provided. They can avoid forming the full KRP explicitly and computing the corresponding probabilities directly. The adaptive step size version of the method is also given. For the proposed method, we present its detailed theoretical properties and comprehensive numerical performance. The results on synthetic and real data show that our method performs better than the corresponding one in the literature.