Reinforcement Learning Algorithms: An Overview and Classification

TL;DR

This paper provides a comprehensive overview and classification of reinforcement learning algorithms based on environment types, aiding practitioners in selecting suitable algorithms for complex real-world problems.

Contribution

It introduces a classification framework for reinforcement learning algorithms according to environment types and analyzes relationships among algorithms within each category.

Findings

Classifies reinforcement learning algorithms into three main environment types.

Provides insights into the foundations and differences of various algorithms.

Helps in selecting appropriate algorithms for specific real-world applications.

Abstract

The desire to make applications and machines more intelligent and the aspiration to enable their operation without human interaction have been driving innovations in neural networks, deep learning, and other machine learning techniques. Although reinforcement learning has been primarily used in video games, recent advancements and the development of diverse and powerful reinforcement algorithms have enabled the reinforcement learning community to move from playing video games to solving complex real-life problems in autonomous systems such as self-driving cars, delivery drones, and automated robotics. Understanding the environment of an application and the algorithms' limitations plays a vital role in selecting the appropriate reinforcement learning algorithm that successfully solves the problem on hand in an efficient manner. Consequently, in this study, we identify three main environment types and classify reinforcement learning algorithms according to those environment types. Moreover, within each category, we identify relationships between algorithms. The overview of each algorithm provides insight into the algorithms' foundations and reviews similarities and differences among algorithms. This study provides a perspective on the field and helps practitioners and researchers to select the appropriate algorithm for their use case.