A linesearch-based derivative-free method for noisy black-box problems

TL;DR

This paper introduces a derivative-free optimization method for noisy black-box problems using linesearch and extrapolation, with proven convergence and complexity bounds.

Contribution

It presents a novel linesearch-based derivative-free algorithm for stochastic black-box optimization with theoretical convergence guarantees.

Findings

Proposed an extrapolation-based derivative-free algorithm.

Proved convergence under reasonable assumptions.

Established worst-case complexity bounds.

Abstract

In this work we consider unconstrained optimization problems. The objective function is known through a zeroth order stochastic oracle that gives an estimate of the true objective function. To solve these problems, we propose a derivative-free algorithm based on extrapolation techniques. Under reasonable assumptions we are able to prove convergence properties for the proposed algorithms. Furthermore, we also give a worst-case complexity result stating that the total number of iterations where the expected value of the norm of the objective function gradient is above a prefixed $\epsilon>0$ is ${\cal O}(n^2\epsilon^{-2}/\beta^2)$ in the worst case.