Local and Global Convergence of General Burer-Monteiro Tensor Optimizations

TL;DR

This paper studies the convergence properties of gradient descent on nonconvex tensor optimization problems reformulated via Burer-Monteiro parameterization, providing local and global geometric insights.

Contribution

It offers the first comprehensive analysis of both local convergence and global geometry for Burer-Monteiro tensor optimization, including conditions for convergence and the landscape of rank-one tensor approximation.

Findings

Gradient descent converges linearly near the true tensor factors.

Orthogonally decomposable tensors have no spurious local minima.

All saddle points are strict except for a third-order saddle at zero.

Abstract

Tensor optimization is crucial to massive machine learning and signal processing tasks. In this paper, we consider tensor optimization with a convex and well-conditioned objective function and reformulate it into a nonconvex optimization using the Burer-Monteiro type parameterization. We analyze the local convergence of applying vanilla gradient descent to the factored formulation and establish a local regularity condition under mild assumptions. We also provide a linear convergence analysis of the gradient descent algorithm started in a neighborhood of the true tensor factors. Complementary to the local analysis, this work also characterizes the global geometry of the best rank-one tensor approximation problem and demonstrates that for orthogonally decomposable tensors the problem has no spurious local minima and all saddle points are strict except for the one at zero which is a third-order saddle point.