Worst case complexity bounds for linesearch-type derivative-free algorithms

TL;DR

This paper establishes that certain linesearch-type derivative-free algorithms for non-convex smooth functions have worst-case complexity bounds similar to pattern search methods, requiring O(ε^{-2}) evaluations to reach a gradient norm below ε.

Contribution

It proves that two linesearch-based derivative-free algorithms share the same worst-case complexity bounds as pattern and direct search algorithms for non-convex optimization.

Findings

Both algorithms require O(ε^{-2}) iterations to achieve a gradient norm below ε.

The complexity bounds match those of pattern and direct search algorithms.

The analysis applies to general non-convex smooth functions.

Abstract

This paper is devoted to the analysis of worst case complexity bounds for linesearch-type derivative-free algorithms for the minimization of general non-convex smooth functions. We prove that two linesearch-type algorithms enjoy the same complexity properties which have been proved for pattern and direct search algorithms. In particular, we consider two derivative-free algorithms based on two different linesearch techniques and manage to prove that the number of iterations and of function evaluations required to drive the norm of the gradient of the objective function below a given threshold $\epsilon$ is ${\cal O}(\epsilon^{-2})$ in the worst case.