A Survey of State Representation Learning for Deep Reinforcement Learning

TL;DR

This survey reviews various state representation learning methods in deep reinforcement learning, categorizing them into six classes, and discusses their mechanisms, benefits, limitations, and evaluation techniques to guide future research.

Contribution

It provides a comprehensive taxonomy of state representation learning methods in model-free online reinforcement learning, clarifying their differences and guiding new researchers.

Findings

Six main classes of methods identified

Evaluation techniques for representation quality discussed

Guidance for future research provided

Abstract

Representation learning methods are an important tool for addressing the challenges posed by complex observations spaces in sequential decision making problems. Recently, many methods have used a wide variety of types of approaches for learning meaningful state representations in reinforcement learning, allowing better sample efficiency, generalization, and performance. This survey aims to provide a broad categorization of these methods within a model-free online setting, exploring how they tackle the learning of state representations differently. We categorize the methods into six main classes, detailing their mechanisms, benefits, and limitations. Through this taxonomy, our aim is to enhance the understanding of this field and provide a guide for new researchers. We also discuss techniques for assessing the quality of representations, and detail relevant future directions.