Single-loop $\mathcal{O}(\epsilon^{-3})$ stochastic smoothing algorithms for nonsmooth Riemannian optimization

TL;DR

This paper introduces two single-loop stochastic smoothing algorithms for nonsmooth Riemannian optimization that achieve optimal iteration complexity and are efficient in terms of per-iteration cost, applicable to a broad class of problems.

Contribution

The paper develops novel Riemannian stochastic smoothing algorithms with optimal complexity and low per-iteration cost, extending results to nonsmooth and constrained problems on manifolds.

Findings

Achieves () iteration complexity for Lipschitz continuous nonsmooth functions.

Designs a new algorithm with () complexity for indicator functions of convex sets.

Framework recovers or matches optimal complexity in various problem settings.

Abstract

In this paper, we develop two Riemannian stochastic smoothing algorithms for nonsmooth optimization problems on Riemannian manifolds, addressing distinct forms of the nonsmooth term \( h \). Both methods combine dynamic smoothing with a momentum-based variance reduction scheme in a fully online manner. When \( h \) is Lipschitz continuous, we propose an stochastic algorithm under adaptive parameter that achieves the optimal iteration complexity of \( \mathcal{O}(\epsilon^{-3}) \), improving upon the best-known rates for exist algorithms. When \( h \) is the indicator function of a convex set, we design a new algorithm using truncated momentum, and under a mild error bound condition with parameter \( \theta \geq 1 \), we establish a complexity of \( \tilde{\mathcal{O}}(\epsilon^{-\max\{\theta+2, 2\theta\}}) \), in line with the best-known results in the Euclidean setting. Both algorithms feature a single-loop design with low per-iteration cost and require only \( \mathcal{O}(1) \) samples per iteration, ensuring that sample and iteration complexities coincide. Our framework encompasses a broad class of problems and recovers or matches optimal complexity guarantees in several important settings, including smooth stochastic Riemannian optimization, composite problems in Euclidean space, and constrained optimization via indicator functions.