Deep Reinforcement Learning Versus Evolution Strategies: A Comparative Survey

TL;DR

This survey compares Deep Reinforcement Learning and Evolution Strategies, analyzing their strengths, limitations, and hybrid approaches, and reviews their applications in real-world problems.

Contribution

It provides a comprehensive comparison of DRL and ESs, highlighting their capabilities, limitations, and potential for hybridization in various applications.

Findings

DRL and ESs have complementary strengths and weaknesses.

Hybrid algorithms can leverage advantages of both methods.

Survey of real-world applications demonstrates practical effectiveness.

Abstract

Deep Reinforcement Learning (DRL) and Evolution Strategies (ESs) have surpassed human-level control in many sequential decision-making problems, yet many open challenges still exist. To get insights into the strengths and weaknesses of DRL versus ESs, an analysis of their respective capabilities and limitations is provided. After presenting their fundamental concepts and algorithms, a comparison is provided on key aspects such as scalability, exploration, adaptation to dynamic environments, and multi-agent learning. Then, the benefits of hybrid algorithms that combine concepts from DRL and ESs are highlighted. Finally, to have an indication about how they compare in real-world applications, a survey of the literature for the set of applications they support is provided.