Riemannian Smoothing Gradient Type Algorithms]{Riemannian Smoothing Gradient Type Algorithms for Nonsmooth Optimization Problem on Compact Riemannian Submanifold Embedded in Euclidean Space

TL;DR

This paper introduces Riemannian smoothing gradient algorithms for nonconvex, nonsmooth optimization on compact Riemannian submanifolds, achieving optimal iteration complexity bounds and validated by numerical experiments.

Contribution

The paper develops the first Riemannian smoothing gradient methods with proven optimal iteration complexity for nonconvex nonsmooth problems on manifolds.

Findings

Riemannian smoothing gradient method has complexity $\\mathcal{O}(\epsilon^{-3})$.

Riemannian smoothing stochastic gradient method has complexity $\\mathcal{O}(\epsilon^{-5})$.

Numerical experiments demonstrate the effectiveness of the proposed algorithms.

Abstract

In this paper, we introduce the notion of generalized $\epsilon$-stationarity for a class of nonconvex and nonsmooth composite minimization problems on compact Riemannian submanifold embedded in Euclidean space. To find a generalized $\epsilon$-stationarity point, we develop a family of Riemannian gradient-type methods based on the Moreau envelope technique with a decreasing sequence of smoothing parameters, namely Riemannian smoothing gradient and Riemannian smoothing stochastic gradient methods. We prove that the Riemannian smoothing gradient method has the iteration complexity of $\mathcal{O}(\epsilon^{-3})$ for driving a generalized $\epsilon$-stationary point. To our knowledge, this is the best-known iteration complexity result for the nonconvex and nonsmooth composite problem on manifolds. For the Riemannian smoothing stochastic gradient method, one can achieve the iteration complexity of $\mathcal{O}(\epsilon^{-5})$ for driving a generalized $\epsilon$-stationary point. Numerical experiments are conducted to validate the superiority of our algorithms.