A derivative-free optimization algorithm for the efficient minimization of functions obtained via statistical averaging

TL;DR

This paper introduces $oldsymbol{ extalpha}$-DOGS, a derivative-free optimization algorithm that adaptively adjusts sampling to efficiently minimize the infinite time average of ergodic processes, reducing costs in expensive physical or numerical experiments.

Contribution

The paper presents $oldsymbol{ extalpha}$-DOGS, a novel adaptive sampling algorithm that improves efficiency in minimizing ergodic process averages, with proven convergence under certain conditions.

Findings

Significant reduction in optimization costs.

Effective adaptive sampling improves accuracy.

Proven convergence to the global minimum.

Abstract

This paper considers the efficient minimization of the infinite time average of a stationary ergodic process in the space of a handful of design parameters which affect it. Problems of this class, derived from physical or numerical experiments which are sometimes expensive to perform, are ubiquitous in engineering applications. In such problems, any given function evaluation, determined with finite sampling, is associated with a quantifiable amount of uncertainty, which may be reduced via additional sampling. The present paper proposes a new optimization algorithm to adjust the amount of sampling associated with each function evaluation, making function evaluations more accurate (and, thus, more expensive), as required, as convergence is approached. The work builds on our algorithm for Delaunay-based Derivative-free Optimization via Global Surrogates ($\Delta$-DOGS). The new algorithm, dubbed $\alpha$-DOGS, substantially reduces the overall cost of the optimization process for problems of this important class. Further, under certain well-defined conditions, rigorous proof of convergence to the global minimum of the problem considered is established.