Pseudorehearsal in actor-critic agents

TL;DR

This paper investigates whether pseudorehearsal techniques can mitigate catastrophic forgetting in actor-critic reinforcement learning agents, specifically in a pole balancing task, and compares different pseudorehearsal methods to enhance learning performance.

Contribution

It introduces the application of pseudorehearsal to actor-critic agents in reinforcement learning and compares various approaches to improve learning stability.

Findings

Pseudorehearsal can help prevent catastrophic forgetting in simple RL tasks.

Proper initialization of rehearsal parameters is crucial for effectiveness.

Different pseudorehearsal methods vary in performance.

Abstract

Catastrophic forgetting has a serious impact in reinforcement learning, as the data distribution is generally sparse and non-stationary over time. The purpose of this study is to investigate whether pseudorehearsal can increase performance of an actor-critic agent with neural-network based policy selection and function approximation in a pole balancing task and compare different pseudorehearsal approaches. We expect that pseudorehearsal assists learning even in such very simple problems, given proper initialization of the rehearsal parameters.