Pseudorehearsal in value function approximation

TL;DR

This paper investigates pseudorehearsal techniques to mitigate catastrophic forgetting in reinforcement learning, demonstrating their effectiveness in a pole balancing task with proper parameter initialization.

Contribution

It compares various pseudorehearsal methods for Q-learning with function approximation, highlighting their benefits in simple reinforcement learning tasks.

Findings

Pseudorehearsal aids learning in non-stationary data environments.

Proper initialization of rehearsal parameters is crucial for effectiveness.

Pseudorehearsal shows promise even in simple RL problems.

Abstract

Catastrophic forgetting is of special importance in reinforcement learning, as the data distribution is generally non-stationary over time. We study and compare several pseudorehearsal approaches for Q-learning with function approximation in a pole balancing task. We have found that pseudorehearsal seems to assist learning even in such very simple problems, given proper initialization of the rehearsal parameters.