Retraction based Direct Search Methods for Derivative Free Riemannian Optimization

TL;DR

This paper extends direct search methods to Riemannian manifolds for black-box optimization, introducing retraction-based algorithms for smooth and nonsmooth functions with convergence guarantees.

Contribution

It introduces the first retraction-based direct search algorithms for nonsmooth Riemannian optimization without derivatives, with theoretical convergence analysis.

Findings

Algorithms successfully minimize nonsmooth functions on manifolds.

Convergence guarantees are established for the proposed methods.

Numerical experiments demonstrate effectiveness on benchmark problems.

Abstract

Direct search methods represent a robust and reliable class of algorithms for solving black-box optimization problems. In this paper, we explore the application of those strategies to Riemannian optimization, wherein minimization is to be performed with respect to variables restricted to lie on a manifold. More specifically, we consider classic and line search extrapolated variants of direct search, and, by making use of retractions, we devise tailored strategies for the minimization of both smooth and nonsmooth functions. As such we analyze, for the first time in the literature, a class of retraction based algorithms for minimizing nonsmooth objectives on a Riemannian manifold without having access to (sub)derivatives. Along with convergence guarantees we provide a set of numerical performance illustrations on a standard set of problems.