Direct Mutation and Crossover in Genetic Algorithms Applied to Reinforcement Learning Tasks

TL;DR

This paper introduces a simple genetic algorithm for neuroevolution in reinforcement learning, with two novel modifications that enhance data efficiency and convergence speed, demonstrated on the FrozenLake environment.

Contribution

It presents two new modifications to genetic algorithms that improve neuroevolution's efficiency and speed in RL tasks.

Findings

Modified GA outperforms baseline in data efficiency

Faster convergence on FrozenLake environment

Significant improvement over initial implementation

Abstract

Neuroevolution has recently been shown to be quite competitive in reinforcement learning (RL) settings, and is able to alleviate some of the drawbacks of gradient-based approaches. This paper will focus on applying neuroevolution using a simple genetic algorithm (GA) to find the weights of a neural network that produce optimally behaving agents. In addition, we present two novel modifications that improve the data efficiency and speed of convergence when compared to the initial implementation. The modifications are evaluated on the FrozenLake environment provided by OpenAI gym and prove to be significantly better than the baseline approach.