Emergence of Novelty in Evolutionary Algorithms

TL;DR

This paper introduces a simple reward-based method to promote behavioral diversity in evolutionary algorithms, effectively avoiding local minima and improving performance across maze and Atari game tasks.

Contribution

The authors propose a novel, simpler approach to encourage diversity in evolutionary algorithms, outperforming previous methods like Novelty Search in various tasks.

Findings

Improved performance in maze problem

Comparable results on Atari Games with less computation

Simpler method than existing diversity techniques

Abstract

One of the main problems of evolutionary algorithms is the convergence of the population to local minima. In this paper, we explore techniques that can avoid this problem by encouraging a diverse behavior of the agents through a shared reward system. The rewards are randomly distributed in the environment, and the agents are only rewarded for collecting them first. This leads to an emergence of a novel behavior of the agents. We introduce our approach to the maze problem and compare it to the previously proposed solution, denoted as Novelty Search (Lehman and Stanley, 2011a). We find that our solution leads to an improved performance while being significantly simpler. Building on that, we generalize the problem and apply our approach to a more advanced set of tasks, Atari Games, where we observe a similar performance quality with much less computational power needed.