Proximal methods for structured nonsmooth optimization over Riemannian submanifolds

TL;DR

This paper introduces a manifold proximal-gradient-subgradient algorithm for structured nonsmooth optimization over Riemannian submanifolds, with convergence guarantees and applications in machine learning.

Contribution

It proposes a novel algorithm for nonsmooth Riemannian optimization problems and establishes convergence under mild conditions, including the Kurdyka-Łojasiewicz property.

Findings

Algorithm converges to critical points under mild conditions.

Enhanced algorithm guarantees convergence to B-stationary points.

Preliminary experiments demonstrate efficiency of the methods.

Abstract

In this paper, we consider a class of structured nonsmooth optimization problems over an embedded submanifold of a Euclidean space, where the first part of the objective is the sum of a difference-of-convex (DC) function and a smooth function, while the remaining part is a weakly convex function over a smooth function. This model problem has many important applications in machine learning and scientific computing, for example, the sparse Fisher discriminant analysis. We propose a manifold proximal-gradient-subgradient algorithm (MPGSA) and show that under mild conditions any accumulation point of the solution sequence generated by it is a critical point of the underlying problem. By assuming the Kurdyka-{\L}ojasiewicz property of an auxiliary function, we further establish the convergence of the full sequence generated by MPGSA under some suitable conditions. When the second component of the DC function involved is the maximum of finite continuously differentiable convex functions, we also propose an enhanced MPGSA with guaranteed subsequential convergence to a lifted B-stationary points of the optimization problem. Finally, some preliminary numerical experiments are conducted to illustrate the efficiency of the proposed algorithms.