Composing Dextrous Grasping and In-hand Manipulation via Scoring with a Reinforcement Learning Critic

TL;DR

This paper introduces a method that uses a reinforcement learning critic to select initial grasps, significantly improving in-hand manipulation success rates and enabling autonomous object reorientation in real-world robotics.

Contribution

It presents a novel approach to select stable initial grasps using the RL critic, bridging the gap between grasping and manipulation without extra training.

Findings

Increased success rate of in-hand manipulation with the proposed method.

Demonstrated autonomous grasping and reorientation of complex objects.

Effective real-world implementation on a robotic system.

Abstract

In-hand manipulation and grasping are fundamental yet often separately addressed tasks in robotics. For deriving in-hand manipulation policies, reinforcement learning has recently shown great success. However, the derived controllers are not yet useful in real-world scenarios because they often require a human operator to place the objects in suitable initial (grasping) states. Finding stable grasps that also promote the desired in-hand manipulation goal is an open problem. In this work, we propose a method for bridging this gap by leveraging the critic network of a reinforcement learning agent trained for in-hand manipulation to score and select initial grasps. Our experiments show that this method significantly increases the success rate of in-hand manipulation without requiring additional training. We also present an implementation of a full grasp manipulation pipeline on a real-world system, enabling autonomous grasping and reorientation even of unwieldy objects.