Emerging Extrinsic Dexterity in Cluttered Scenes via Dynamics-aware Policy Learning

TL;DR

This paper introduces a Dynamics-Aware Policy Learning framework that enables robots to perform extrinsic dexterity in cluttered scenes by modeling contact-induced object dynamics, leading to improved success rates in simulation and real-world tasks.

Contribution

The paper presents a novel framework that explicitly models contact dynamics for policy learning, enabling extrinsic dexterity without hand-crafted heuristics or reward shaping.

Findings

Outperforms prior methods by over 25% in success rate in simulation.

Achieves around 50% success rate in real-world cluttered scenes.

Demonstrates robust sim-to-real transfer in practical grocery tasks.

Abstract

Extrinsic dexterity leverages environmental contact to overcome the limitations of prehensile manipulation. However, achieving such dexterity in cluttered scenes remains challenging and underexplored, as it requires selectively exploiting contact among multiple interacting objects with inherently coupled dynamics. Existing approaches lack explicit modeling of such complex dynamics and therefore fall short in non-prehensile manipulation in cluttered environments, which in turn limits their practical applicability in real-world environments. In this paper, we introduce a Dynamics-Aware Policy Learning (DAPL) framework that can facilitate policy learning with a learned representation of contact-induced object dynamics in cluttered environments. This representation is learned through explicit world modeling and used to condition reinforcement learning, enabling extrinsic dexterity to emerge without hand-crafted contact heuristics or complex reward shaping. We evaluate our approach in both simulation and the real world. Our method outperforms prehensile manipulation, human teleoperation, and prior representation-based policies by over 25% in success rate on unseen simulated cluttered scenes with varying densities. The real-world success rate reaches around 50% across 10 cluttered scenes, while a practical grocery deployment further demonstrates robust sim-to-real transfer and applicability.