A Survey of Deep Network Solutions for Learning Control in Robotics: From Reinforcement to Imitation

TL;DR

This survey reviews deep learning methods for robotic control, focusing on deep reinforcement learning and imitation learning, highlighting their applications, challenges, and transfer to real-world scenarios.

Contribution

It provides a comprehensive overview of deep learning solutions for robotic control, detailing algorithms, applications, and transfer challenges in a unified survey.

Findings

DRL effectively addresses navigation and manipulation tasks.

Simulation-to-real transfer remains a key challenge.

Imitation learning categories are well-suited for various robotic tasks.

Abstract

Deep learning techniques have been widely applied, achieving state-of-the-art results in various fields of study. This survey focuses on deep learning solutions that target learning control policies for robotics applications. We carry out our discussions on the two main paradigms for learning control with deep networks: deep reinforcement learning and imitation learning. For deep reinforcement learning (DRL), we begin from traditional reinforcement learning algorithms, showing how they are extended to the deep context and effective mechanisms that could be added on top of the DRL algorithms. We then introduce representative works that utilize DRL to solve navigation and manipulation tasks in robotics. We continue our discussion on methods addressing the challenge of the reality gap for transferring DRL policies trained in simulation to real-world scenarios, and summarize robotics simulation platforms for conducting DRL research. For imitation leaning, we go through its three main categories, behavior cloning, inverse reinforcement learning and generative adversarial imitation learning, by introducing their formulations and their corresponding robotics applications. Finally, we discuss the open challenges and research frontiers.