Efficient Low-Tubal-Rank Tensor Estimation via Alternating Preconditioned Gradient Descent

TL;DR

This paper introduces an Accelerated Preconditioned Gradient Descent (APGD) method for low-tubal-rank tensor estimation, achieving faster convergence even when the tensor rank is overestimated, with theoretical guarantees and empirical validation.

Contribution

The paper proposes a novel APGD algorithm that accelerates convergence in over-parameterized low-tubal-rank tensor estimation, with proven linear convergence and independence from tensor condition number.

Findings

APGD achieves linear convergence in tensor estimation.

Convergence rate is independent of tensor condition number.

Empirical results validate theoretical guarantees.

Abstract

The problem of low-tubal-rank tensor estimation is a fundamental task with wide applications across high-dimensional signal processing, machine learning, and image science. Traditional approaches tackle such a problem by performing tensor singular value decomposition, which is computationally expensive and becomes infeasible for large-scale tensors. Recent approaches address this issue by factorizing the tensor into two smaller factor tensors and solving the resulting problem using gradient descent. However, this kind of approach requires an accurate estimate of the tensor rank, and when the rank is overestimated, the convergence of gradient descent and its variants slows down significantly or even diverges. To address this problem, we propose an Alternating Preconditioned Gradient Descent (APGD) algorithm, which accelerates convergence in the over-parameterized setting by adding a preconditioning term to the original gradient and updating these two factors alternately. Based on certain geometric assumptions on the objective function, we establish linear convergence guarantees for more general low-tubal-rank tensor estimation problems. Then we further analyze the specific cases of low-tubal-rank tensor factorization and low-tubal-rank tensor recovery. Our theoretical results show that APGD achieves linear convergence even under over-parameterization, and the convergence rate is independent of the tensor condition number. Extensive simulations on synthetic data are carried out to validate our theoretical assertions.