QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation

TL;DR

This paper introduces QT-Opt, a scalable deep reinforcement learning framework that enables vision-based, closed-loop robotic grasping with high success rates and adaptive behaviors, trained on extensive real-world data.

Contribution

We present QT-Opt, a novel scalable reinforcement learning approach that trains a deep neural network for vision-based robotic grasping, enabling dynamic, closed-loop control and complex manipulation behaviors.

Findings

Achieved 96% grasp success on unseen objects.

Learned regrasping and repositioning strategies automatically.

Demonstrated robustness to disturbances and perturbations.

Abstract

In this paper, we study the problem of learning vision-based dynamic manipulation skills using a scalable reinforcement learning approach. We study this problem in the context of grasping, a longstanding challenge in robotic manipulation. In contrast to static learning behaviors that choose a grasp point and then execute the desired grasp, our method enables closed-loop vision-based control, whereby the robot continuously updates its grasp strategy based on the most recent observations to optimize long-horizon grasp success. To that end, we introduce QT-Opt, a scalable self-supervised vision-based reinforcement learning framework that can leverage over 580k real-world grasp attempts to train a deep neural network Q-function with over 1.2M parameters to perform closed-loop, real-world grasping that generalizes to 96% grasp success on unseen objects. Aside from attaining a very high success rate, our method exhibits behaviors that are quite distinct from more standard grasping systems: using only RGB vision-based perception from an over-the-shoulder camera, our method automatically learns regrasping strategies, probes objects to find the most effective grasps, learns to reposition objects and perform other non-prehensile pre-grasp manipulations, and responds dynamically to disturbances and perturbations.