Proximal Quasi-Newton Method for Composite Optimization over the Stiefel Manifold

TL;DR

This paper introduces the ManPQN method, a Riemannian proximal quasi-Newton algorithm designed for composite optimization on the Stiefel manifold, demonstrating improved convergence and efficiency over existing methods.

Contribution

The paper develops a novel Riemannian proximal quasi-Newton method with proven global convergence and local linear convergence for optimization on the Stiefel manifold, extending Euclidean quasi-Newton techniques.

Findings

ManPQN accelerates convergence compared to proximal gradient methods.

Global convergence of ManPQN is theoretically established.

Numerical results show improved efficiency in practical problems.

Abstract

In this paper, we consider the composite optimization problems over the Stiefel manifold. A successful method to solve this class of problems is the proximal gradient method proposed by Chen et al. Motivated by the proximal Newton-type techniques in the Euclidean space, we present a Riemannian proximal quasi-Newton method, named ManPQN, to solve the composite optimization problems. The global convergence of the ManPQN method is proved and iteration complexity for obtaining an $\epsilon$-stationary point is analyzed. Under some mild conditions, we also establish the local linear convergence result of the ManPQN method. Numerical results are encouraging, which shows that the proximal quasi-Newton technique can be used to accelerate the proximal gradient method.