Evolution Strategies Converges to Finite Differences

TL;DR

This paper demonstrates that the gradient estimates of Evolution Strategies (ES) and Finite Differences (FD) become similar as the problem dimension grows, clarifying their relationship in high-dimensional optimization.

Contribution

It provides a formal proof that ES gradients converge to FD gradients in high-dimensional settings, establishing a theoretical link between the two methods.

Findings

ES and FD gradients converge as dimension increases

The difference between ES and FD diminishes in high-dimensional spaces

Theoretical analysis clarifies the relationship between ES and FD gradients

Abstract

Since the debut of Evolution Strategies (ES) as a tool for Reinforcement Learning by Salimans et al. 2017, there has been interest in determining the exact relationship between the Evolution Strategies gradient and the gradient of a similar class of algorithms, Finite Differences (FD).(Zhang et al. 2017, Lehman et al. 2018) Several investigations into the subject have been performed, investigating the formal motivational differences(Lehman et al. 2018) between ES and FD, as well as the differences in a standard benchmark problem in Machine Learning, the MNIST classification problem(Zhang et al. 2017). This paper proves that while the gradients are different, they converge as the dimension of the vector under optimization increases.