Challenges of Real-World Reinforcement Learning

TL;DR

This paper identifies nine key challenges for deploying reinforcement learning in real-world applications, discusses existing approaches, and proposes a testbed for practical RL research to address these issues.

Contribution

It systematically defines real-world RL challenges, reviews literature solutions, and introduces a testbed for evaluating approaches in practical scenarios.

Findings

Nine challenges are critical for real-world RL deployment

Existing approaches partially address these challenges

A testbed is proposed for practical RL research

Abstract

Reinforcement learning (RL) has proven its worth in a series of artificial domains, and is beginning to show some successes in real-world scenarios. However, much of the research advances in RL are often hard to leverage in real-world systems due to a series of assumptions that are rarely satisfied in practice. We present a set of nine unique challenges that must be addressed to productionize RL to real world problems. For each of these challenges, we specify the exact meaning of the challenge, present some approaches from the literature, and specify some metrics for evaluating that challenge. An approach that addresses all nine challenges would be applicable to a large number of real world problems. We also present an example domain that has been modified to present these challenges as a testbed for practical RL research.