Geometry-aware RL for Manipulation of Varying Shapes and Deformable Objects

TL;DR

This paper introduces a geometry-aware reinforcement learning framework using heterogeneous graph representations and equivariant neural networks to improve manipulation of varying shapes and deformable objects in robotics.

Contribution

It presents a novel graph-based policy model, HEPi, that leverages $SE(3)$ equivariance and heterogeneity to enhance manipulation tasks involving complex geometries and deformable objects.

Findings

HEPi outperforms Transformer-based policies in average returns.

HEPi demonstrates better sample efficiency.

HEPi generalizes well to unseen objects.

Abstract

Manipulating objects with varying geometries and deformable objects is a major challenge in robotics. Tasks such as insertion with different objects or cloth hanging require precise control and effective modelling of complex dynamics. In this work, we frame this problem through the lens of a heterogeneous graph that comprises smaller sub-graphs, such as actuators and objects, accompanied by different edge types describing their interactions. This graph representation serves as a unified structure for both rigid and deformable objects tasks, and can be extended further to tasks comprising multiple actuators. To evaluate this setup, we present a novel and challenging reinforcement learning benchmark, including rigid insertion of diverse objects, as well as rope and cloth manipulation with multiple end-effectors. These tasks present a large search space, as both the initial and target configurations are uniformly sampled in 3D space. To address this issue, we propose a novel graph-based policy model, dubbed Heterogeneous Equivariant Policy (HEPi), utilizing $SE(3)$ equivariant message passing networks as the main backbone to exploit the geometric symmetry. In addition, by modeling explicit heterogeneity, HEPi can outperform Transformer-based and non-heterogeneous equivariant policies in terms of average returns, sample efficiency, and generalization to unseen objects. Our project page is available at https://thobotics.github.io/hepi.