An Inexact Riemannian Proximal Gradient Method

TL;DR

This paper introduces an inexact Riemannian proximal gradient method that guarantees convergence under practical accuracy conditions, enabling efficient solutions for optimization problems on manifolds like sparse PCA.

Contribution

It develops a convergence analysis for an inexact Riemannian proximal gradient method, relaxing the need for exact solutions of the proximal mapping, with practical accuracy conditions.

Findings

Proven global convergence under inexact proximal solutions.

Established local convergence rate based on Riemannian Kurdyka-ojasiewicz property.

Numerical experiments validate practical accuracy conditions on sparse PCA.

Abstract

This paper considers the problem of minimizing the summation of a differentiable function and a nonsmooth function on a Riemannian manifold. In recent years, proximal gradient method and its invariants have been generalized to the Riemannian setting for solving such problems. Different approaches to generalize the proximal mapping to the Riemannian setting lead versions of Riemannian proximal gradient methods. However, their convergence analyses all rely on solving their Riemannian proximal mapping exactly, which is either too expensive or impracticable. In this paper, we study the convergence of an inexact Riemannian proximal gradient method. It is proven that if the proximal mapping is solved sufficiently accurately, then the global convergence and local convergence rate based on the Riemannian Kurdyka-\L ojasiewicz property can be guaranteed. Moreover, practical conditions on the accuracy for solving the Riemannian proximal mapping are provided. As a byproduct, the proximal gradient method on the Stiefel manifold proposed in~[CMSZ2020] can be viewed as the inexact Riemannian proximal gradient method provided the proximal mapping is solved to certain accuracy. Finally, numerical experiments on sparse principal component analysis are conducted to test the proposed practical conditions.