Curiosity creates Diversity in Policy Search
Paul-Antoine Le Tolguenec, Emmanuel Rachelson, Yann Besse, Dennis G., Wilson

TL;DR
This paper introduces Curiosity-ES, an evolutionary strategy that uses intrinsic motivation to promote diversity in policy search, especially in reward-sparse environments, leading to multiple reward-yielding policies.
Contribution
It proposes a novel evolutionary strategy that incorporates Curiosity as a fitness metric, outperforming traditional diversity metrics like Novelty.
Findings
Curiosity-ES generates higher diversity without explicit diversity criteria.
Curiosity-ES finds multiple reward-yielding policies.
Curiosity outperforms Novelty in promoting diverse behaviors.
Abstract
When searching for policies, reward-sparse environments often lack sufficient information about which behaviors to improve upon or avoid. In such environments, the policy search process is bound to blindly search for reward-yielding transitions and no early reward can bias this search in one direction or another. A way to overcome this is to use intrinsic motivation in order to explore new transitions until a reward is found. In this work, we use a recently proposed definition of intrinsic motivation, Curiosity, in an evolutionary policy search method. We propose Curiosity-ES, an evolutionary strategy adapted to use Curiosity as a fitness metric. We compare Curiosity with Novelty, a commonly used diversity metric, and find that Curiosity can generate higher diversity over full episodes without the need for an explicit diversity criterion and lead to multiple policies which find reward.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsEvolution and Genetic Dynamics · Evolutionary Game Theory and Cooperation · Experimental Behavioral Economics Studies
