Derivative-free global minimization for a class of multiple minima problems

TL;DR

This paper proves that a derivative-free method using finite differences can globally minimize certain classes of functions with multiple minima, achieving linear convergence and low computational complexity, supported by numerical experiments.

Contribution

It introduces and analyzes the FD-DFD method, showing its ability to find global minima with linear convergence for extended strongly convex functions.

Findings

FD-DFD converges linearly to the global minimizer.

Per-iteration cost is nearly independent of problem dimension.

Numerical experiments confirm efficiency across various dimensions.

Abstract

We prove that the finite-difference based derivative-free descent (FD-DFD) methods have a capability to find the global minima for a class of multiple minima problems. Our main result shows that, for a class of multiple minima objectives that is extended from strongly convex functions with Lipschitz-continuous gradients, the iterates of FD-DFD converge to the global minimizer $x_*$ with the linear convergence $\|x_{k+1}-x_*\|_2^2\leqslant\rho^k \|x_1-x_*\|_2^2$ for a fixed $0<\rho<1$ and any initial iteration $x_1\in\mathbb{R}^d$ when the parameters are properly selected. Since the per-iteration cost, i.e., the number of function evaluations, is fixed and almost independent of the dimension $d$, the FD-DFD algorithm has a complexity bound $\mathcal{O}(\log\frac{1}{\epsilon})$ for finding a point $x$ such that the optimality gap $\|x-x_*\|_2^2$ is less than $\epsilon>0$. Numerical experiments in various dimensions from $5$ to $500$ demonstrate the benefits of the FD-DFD method.