A Cross Entropy based Optimization Algorithm with Global Convergence Guarantees

TL;DR

This paper introduces a new stochastic approximation version of the cross entropy method that improves efficiency and guarantees global convergence, demonstrated through benchmark tests.

Contribution

A novel incremental geometric averaging approach for the cross entropy method that enhances efficiency and ensures global convergence.

Findings

Reduces computational and storage costs significantly.

Achieves convergence to the global optimum for certain functions.

Validated effectiveness on various benchmark problems.

Abstract

The cross entropy (CE) method is a model based search method to solve optimization problems where the objective function has minimal structure. The Monte-Carlo version of the CE method employs the naive sample averaging technique which is inefficient, both computationally and space wise. We provide a novel stochastic approximation version of the CE method, where the sample averaging is replaced with incremental geometric averaging. This approach can save considerable computational and storage costs. Our algorithm is incremental in nature and possesses additional attractive features such as accuracy, stability, robustness and convergence to the global optimum for a particular class of objective functions. We evaluate the algorithm on a variety of global optimization benchmark problems and the results obtained corroborate our theoretical findings.