Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes

TL;DR

This survey reviews the application of deep reinforcement learning in robotics, highlighting real-world successes, key challenges, and future directions for developing capable, sample-efficient robotic systems.

Contribution

It provides a comprehensive overview of DRL successes in robotics, analyzes factors behind these achievements, and outlines future research avenues for practical deployment.

Findings

DRL has achieved notable real-world robotic competencies.

Challenges include sample efficiency and stability in physical environments.

Future work should focus on holistic, long-horizon, open-world tasks.

Abstract

Reinforcement learning (RL), particularly its combination with deep neural networks referred to as deep RL (DRL), has shown tremendous promise across a wide range of applications, suggesting its potential for enabling the development of sophisticated robotic behaviors. Robotics problems, however, pose fundamental difficulties for the application of RL, stemming from the complexity and cost of interacting with the physical world. This article provides a modern survey of DRL for robotics, with a particular focus on evaluating the real-world successes achieved with DRL in realizing several key robotic competencies. Our analysis aims to identify the key factors underlying those exciting successes, reveal underexplored areas, and provide an overall characterization of the status of DRL in robotics. We highlight several important avenues for future work, emphasizing the need for stable and sample-efficient real-world RL paradigms, holistic approaches for discovering and integrating various competencies to tackle complex long-horizon, open-world tasks, and principled development and evaluation procedures. This survey is designed to offer insights for both RL practitioners and roboticists toward harnessing RL's power to create generally capable real-world robotic systems.