An Adaptive Smoothing Algorithm for Non-Lipschitz Optimization on Manifolds with Complexity Guarantees

TL;DR

This paper introduces a novel smoothing Riemannian gradient algorithm with complexity guarantees for non-Lipschitz optimization problems on manifolds, extending existing techniques to handle quasi-norm penalties.

Contribution

It develops the first complexity analysis for non-Lipschitz Riemannian optimization, proposing a smoothing framework and an AdaGrad-type stepsize rule for improved efficiency.

Findings

Achieves an iteration complexity of O(ε^{p-4}) for non-Lipschitz problems.

Demonstrates global convergence of the proposed algorithm.

Preliminary experiments show practical efficiency in machine learning applications.

Abstract

We study a class of optimization problems on Riemannian manifolds, where the objective function consists of a smooth term and quasi-norm type penalties with exponent $p \in (0, 1]$. The essential difficulty lies in the fact that the objective function may not be locally Lipschitz continuous, which places this type of problems beyond the reach of existing Riemannian techniques. To overcome this obstacle, this paper constructs a general smoothing framework and establishes fundamental properties for developing efficient algorithms. In particular, we propose a smoothing Riemannian gradient algorithm equipped with a smoothing-aware AdaGrad-type stepsize rule. Its global convergence is demonstrated together with an iteration complexity of $O (\epsilon^{p - 4})$, which includes the best available iteration complexity of $O (\epsilon^{- 3})$ for Lipschitz problems with $p = 1$ as a special case. To the best of our knowledge, this is the first complexity result for non-Lipschitz optimization on Riemannian manifolds. Preliminary numerical experiments corroborate the practical efficiency of the proposed approach in real-world applications arsing from machine learning and data science.