Jacobi-type algorithms for homogeneous polynomial optimization on Stiefel manifolds with applications to tensor approximations

TL;DR

This paper introduces gradient-based Jacobi-type algorithms for optimizing homogeneous polynomials on Stiefel manifolds, proving their convergence and demonstrating applications to tensor diagonalization.

Contribution

It develops new Jacobi-type algorithms with proven global convergence for polynomial optimization on Stiefel manifolds, including proximal variants, with applications to tensor approximation.

Findings

Algorithms converge globally under certain conditions

Proposed methods are efficient for tensor diagonalization

Proximal variants have convergence without additional conditions

Abstract

This paper mainly studies the gradient-based Jacobi-type algorithms to maximize two classes of homogeneous polynomials with orthogonality constraints, and establish their convergence properties. For the first class of homogeneous polynomials subject to a constraint on a Stiefel manifold, we reformulate it as an optimization problem on a unitary group, which makes it possible to apply the gradient-based Jacobi-type (Jacobi-G) algorithm. Then, if the subproblem can always be represented as a quadratic form, we establish the global convergence of Jacobi-G under any one of three conditions. The convergence result for the first condition is an easy extension of the result in [Usevich et al. SIOPT 2020], while other two conditions are new ones. This algorithm and the convergence properties apply to the well-known joint approximate symmetric tensor diagonalization. For the second class of homogeneous polynomials subject to constraints on the product of Stiefel manifolds, we reformulate it as an optimization problem on the product of unitary groups, and then develop a new gradient-based multi-block Jacobi-type (Jacobi-MG) algorithm to solve it. We establish the global convergence of Jacobi-MG under any one of the above three conditions, if the subproblem can always be represented as a quadratic form. This algorithm and the convergence properties are suitable to the well-known joint approximate tensor diagonalization. As the proximal variants of Jacobi-G and Jacobi-MG, we also propose the Jacobi-GP and Jacobi-MGP algorithms, and establish their global convergence without any further condition. Some numerical results are provided indicating the efficiency of the proposed algorithms.