Continual Reinforcement Learning via Autoencoder-Driven Task and New Environment Recognition

TL;DR

This paper introduces an autoencoder-based method for continual reinforcement learning that detects new tasks and environments, enabling agents to learn continuously without external signals indicating task changes.

Contribution

The study presents a novel autoencoder-driven approach that integrates environment recognition with policy optimization for continual reinforcement learning.

Findings

Successful recognition of new tasks and environments

Effective knowledge preservation during continual learning

No external signals needed for task change detection

Abstract

Continual learning for reinforcement learning agents remains a significant challenge, particularly in preserving and leveraging existing information without an external signal to indicate changes in tasks or environments. In this study, we explore the effectiveness of autoencoders in detecting new tasks and matching observed environments to previously encountered ones. Our approach integrates policy optimization with familiarity autoencoders within an end-to-end continual learning system. This system can recognize and learn new tasks or environments while preserving knowledge from earlier experiences and can selectively retrieve relevant knowledge when re-encountering a known environment. Initial results demonstrate successful continual learning without external signals to indicate task changes or reencounters, showing promise for this methodology.