Block-Randomized Stochastic Proximal Gradient for Low-Rank Tensor Factorization

TL;DR

This paper introduces a flexible stochastic optimization framework combining randomized block coordinate descent and proximal gradient methods for large-scale dense tensor decomposition, with proven convergence and practical effectiveness.

Contribution

It proposes a novel doubly randomized stochastic proximal gradient algorithm for constrained CPD, addressing dense tensors and incorporating various regularizations.

Findings

Effective on large dense tensors in experiments.

Supports a wide range of regularizers and constraints.

Converges under the proposed framework.

Abstract

This work considers the problem of computing the canonical polyadic decomposition (CPD) of large tensors. Prior works mostly leverage data sparsity to handle this problem, which is not suitable for handling dense tensors that often arise in applications such as medical imaging, computer vision, and remote sensing. Stochastic optimization is known for its low memory cost and per-iteration complexity when handling dense data. However, exisiting stochastic CPD algorithms are not flexible enough to incorporate a variety of constraints/regularizations that are of interest in signal and data analytics. Convergence properties of many such algorithms are also unclear. In this work, we propose a stochastic optimization framework for large-scale CPD with constraints/regularizations. The framework works under a doubly randomized fashion, and can be regarded as a judicious combination of randomized block coordinate descent (BCD) and stochastic proximal gradient (SPG). The algorithm enjoys lightweight updates and small memory footprint. In addition, this framework entails considerable flexibility---many frequently used regularizers and constraints can be readily handled under the proposed scheme. The approach is also supported by convergence analysis. Numerical results on large-scale dense tensors are employed to showcase the effectiveness of the proposed approach.