Deformable Cluster Manipulation via Whole-Arm Policy Learning

TL;DR

This paper introduces a model-free reinforcement learning framework for deformable cluster manipulation using whole-arm contact, improving efficiency and enabling zero-shot transfer from simulation to real-world scenarios.

Contribution

It presents a novel multi-modal policy learning approach with a distributional state representation and a context-agnostic occlusion heuristic for deformable object manipulation.

Findings

Effective in power line clearance tasks with multiple arm strategies

Achieves real-time inference and improved training efficiency

Enables zero-shot sim-to-real transfer for complex deformable manipulation

Abstract

Manipulating clusters of deformable objects presents a substantial challenge with widespread applicability, but requires contact-rich whole-arm interactions. A potential solution must address the limited capacity for realistic model synthesis, high uncertainty in perception, and the lack of efficient spatial abstractions, among others. We propose a novel framework for learning model-free policies integrating two modalities: 3D point clouds and proprioceptive touch indicators, emphasising manipulation with full body contact awareness, going beyond traditional end-effector modes. Our reinforcement learning framework leverages a distributional state representation, aided by kernel mean embeddings, to achieve improved training efficiency and real-time inference. Furthermore, we propose a novel context-agnostic occlusion heuristic to clear deformables from a target region for exposure tasks. We deploy the framework in a power line clearance scenario and observe that the agent generates creative strategies leveraging multiple arm links for de-occlusion. Finally, we perform zero-shot sim-to-real policy transfer, allowing the arm to clear real branches with unknown occlusion patterns, unseen topology, and uncertain dynamics. Website: https://sites.google.com/view/dcmwap/