Asymptotic convergence rates for averaging strategies

TL;DR

This paper extends theoretical analysis of averaging strategies in parallel black box optimization to a broader class of functions, demonstrating improved asymptotic convergence rates over random search through formal proofs and experiments.

Contribution

It generalizes previous quadratic-focused results to three times differentiable functions, providing formal convergence rates for averaging strategies in continuous domains.

Findings

Averaging strategies outperform pure random search asymptotically.

Formal convergence rates are established for a broad class of functions.

Experimental validation confirms theoretical predictions.

Abstract

Parallel black box optimization consists in estimating the optimum of a function using $\lambda$ parallel evaluations of $f$. Averaging the $\mu$ best individuals among the $\lambda$ evaluations is known to provide better estimates of the optimum of a function than just picking up the best. In continuous domains, this averaging is typically just based on (possibly weighted) arithmetic means. Previous theoretical results were based on quadratic objective functions. In this paper, we extend the results to a wide class of functions, containing three times continuously differentiable functions with unique optimum. We prove formal rate of convergences and show they are indeed better than pure random search asymptotically in $\lambda$. We validate our theoretical findings with experiments on some standard black box functions.