A Derivative-Free CoMirror Algorithm

TL;DR

This paper introduces a derivative-free optimization algorithm based on the $ ext{eps}$-comirror method for convex constrained problems, demonstrating convergence and practical feasibility through linear interpolation of subgradients.

Contribution

It develops a derivative-free variant of the $ ext{eps}$-comirror algorithm for convex optimization with convergence guarantees, using linear interpolation to approximate subgradients.

Findings

Algorithm achieves the same convergence as gradient-based methods.

Linear interpolation effectively approximates subgradients in derivative-free setting.

Numerical tests confirm practical applicability of the proposed method.

Abstract

We consider $\min\{f(x):g(x) \le 0, ~x\in X\},$ where $X$ is a compact convex subset of $\RR^m$, and $f$ and $g$ are continuous convex functions defined on an open neighbourhood of $X$. We work in the setting of derivative-free optimization, assuming that $f$ and $g$ are available through a black-box that provides only function values for a lower-$\mathcal{C}^2$ representation of the functions. We present a derivative-free optimization variant of the $\eps$-comirror algorithm \cite{BBTGBT2010}. Algorithmic convergence hinges on the ability to accurately approximate subgradients of lower-$\mathcal{C}^2$ functions, which we prove is possible through linear interpolation. We provide convergence analysis that quantifies the difference between the function values of the iterates and the optimal function value. We find that the DFO algorithm we develop has the same convergence result as the original gradient-based algorithm. We present some numerical testing that demonstrate the practical feasibility of the algorithm, and conclude with some directions for further research.