Retrieval Dexterity: Efficient Object Retrieval in Clutters with Dexterous Hand

TL;DR

This paper introduces a reinforcement learning-based dexterous system that efficiently retrieves objects from cluttered environments by learning manipulation skills, outperforming traditional sequential methods and transferring effectively to real robots.

Contribution

It presents a novel approach using large-scale reinforcement learning to develop manipulation policies that clear clutter efficiently, with successful real-world robot transfer.

Findings

Emergent manipulation skills like pushing and poking improve retrieval efficiency.

Policies outperform sequential grasping methods in cluttered environments.

Successful transfer of learned policies to real-world robotic systems.

Abstract

Retrieving objects buried beneath multiple objects is not only challenging but also time-consuming. Performing manipulation in such environments presents significant difficulty due to complex contact relationships. Existing methods typically address this task by sequentially grasping and removing each occluding object, resulting in lengthy execution times and requiring impractical grasping capabilities for every occluding object. In this paper, we present a dexterous arm-hand system for efficient object retrieval in multi-object stacked environments. Our approach leverages large-scale parallel reinforcement learning within diverse and carefully designed cluttered environments to train policies. These policies demonstrate emergent manipulation skills (e.g., pushing, stirring, and poking) that efficiently clear occluding objects to expose sufficient surface area of the target object. We conduct extensive evaluations across a set of over 10 household objects in diverse clutter configurations, demonstrating superior retrieval performance and efficiency for both trained and unseen objects. Furthermore, we successfully transfer the learned policies to a real-world dexterous multi-fingered robot system, validating their practical applicability in real-world scenarios. Videos can be found on our project website https://ChangWinde.github.io/RetrDex.