Preliminary Tests of the Anticipatory Classifier System with Hindsight Experience Replay
Olgierd Unold, Stanis{\l}aw Franczyk

TL;DR
This paper presents ACS2HER, an innovative integration of Anticipatory Classifier System with Hindsight Experience Replay, significantly improving learning speed in sparse reward environments but with increased computational costs.
Contribution
It introduces a novel architecture combining anticipatory mechanisms with hindsight goal relabeling in Learning Classifier Systems, enhancing learning efficiency in sparse reward settings.
Findings
ACS2HER accelerates knowledge acquisition compared to standard ACS2.
The model performs well on Maze 6 and FrozenLake benchmarks.
Increased computational overhead and classifier growth observed.
Abstract
This paper introduces ACS2HER, a novel integration of the Anticipatory Classifier System (ACS2) with the Hindsight Experience Replay (HER) mechanism. While ACS2 is highly effective at building cognitive maps through latent learning, its performance often stagnates in environments characterized by sparse rewards. We propose a specific architectural variant that triggers hindsight learning when the agent fails to reach its primary goal, re-labeling visited states as virtual goals to densify the learning signal. The proposed model was evaluated on two benchmarks: the deterministic \texttt{Maze 6} and the stochastic \texttt{FrozenLake}. The results demonstrate that ACS2HER significantly accelerates knowledge acquisition and environmental mastery compared to the standard ACS2. However, this efficiency gain is accompanied by increased computational overhead and a substantial expansion in…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsEvolutionary Algorithms and Applications · Neural Networks and Reservoir Computing · Machine Learning and Data Classification
