Combining Evolution and Deep Reinforcement Learning for Policy Search: a Survey

TL;DR

This survey reviews recent methods combining deep neuroevolution and deep reinforcement learning, organizing 45 algorithms into a unified framework to facilitate understanding and future research in this emerging area.

Contribution

It systematically categorizes and analyzes existing combination techniques, providing a comprehensive overview to advance the field.

Findings

Organized 45 recent algorithms into a generic framework.

Highlighted relationships and differences between methods.

Identified gaps and suggested directions for future research.

Abstract

Deep neuroevolution and deep Reinforcement Learning have received a lot of attention in the last years. Some works have compared them, highlighting theirs pros and cons, but an emerging trend consists in combining them so as to benefit from the best of both worlds. In this paper, we provide a survey of this emerging trend by organizing the literature into related groups of works and casting all the existing combinations in each group into a generic framework. We systematically cover all easily available papers irrespective of their publication status, focusing on the combination mechanisms rather than on the experimental results. In total, we cover 45 algorithms more recent than 2017. We hope this effort will favor the growth of the domain by facilitating the understanding of the relationships between the methods, leading to deeper analyses, outlining missing useful comparisons and suggesting new combinations of mechanisms.