Quality with Just Enough Diversity in Evolutionary Policy Search
Paul Templier, Luca Grillotti, Emmanuel Rachelson, Dennis G. Wilson,, Antoine Cully

TL;DR
This paper introduces JEDi, a framework that leverages behavior information to efficiently focus evolutionary policy search, outperforming existing methods on complex exploration and control tasks.
Contribution
JEDi is a novel approach that learns the relationship between behavior and fitness to guide evaluations, improving efficiency and performance in evolutionary policy search.
Findings
JEDi outperforms QD and ES on maze and control tasks.
JEDi efficiently identifies promising search areas.
JEDi improves exploration and policy quality.
Abstract
Evolution Strategies (ES) are effective gradient-free optimization methods that can be competitive with gradient-based approaches for policy search. ES only rely on the total episodic scores of solutions in their population, from which they estimate fitness gradients for their update with no access to true gradient information. However this makes them sensitive to deceptive fitness landscapes, and they tend to only explore one way to solve a problem. Quality-Diversity methods such as MAP-Elites introduced additional information with behavior descriptors (BD) to return a population of diverse solutions, which helps exploration but leads to a large part of the evaluation budget not being focused on finding the best performing solution. Here we show that behavior information can also be leveraged to find the best policy by identifying promising search areas which can then be efficiently…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsAgricultural Innovations and Practices · Complex Systems and Decision Making · Evolution and Genetic Dynamics
MethodsFocus
