Lineage Evolution Reinforcement Learning

TL;DR

This paper introduces a lineage evolution reinforcement learning algorithm that combines genetic operations with reinforcement learning to enhance agent performance, demonstrated on Atari 2600 games.

Contribution

It presents a novel lineage-based evolutionary framework integrated with reinforcement learning, improving performance without altering original algorithm parameters.

Findings

Enhanced agent performance in Atari 2600 games

Effective integration of genetic operations with RL algorithms

Lineage consideration improves learning stability

Abstract

We propose a general agent population learning system, and on this basis, we propose lineage evolution reinforcement learning algorithm. Lineage evolution reinforcement learning is a kind of derivative algorithm which accords with the general agent population learning system. We take the agents in DQN and its related variants as the basic agents in the population, and add the selection, mutation and crossover modules in the genetic algorithm to the reinforcement learning algorithm. In the process of agent evolution, we refer to the characteristics of natural genetic behavior, add lineage factor to ensure the retention of potential performance of agent, and comprehensively consider the current performance and lineage value when evaluating the performance of agent. Without changing the parameters of the original reinforcement learning algorithm, lineage evolution reinforcement learning can optimize different reinforcement learning algorithms. Our experiments show that the idea of evolution with lineage improves the performance of original reinforcement learning algorithm in some games in Atari 2600.