Learning Equivariant Neural-Augmented Object Dynamics From Few Interactions

TL;DR

PIEGraph combines physics-based models and equivariant graph neural networks to learn object dynamics efficiently from limited data, improving robotic manipulation of both rigid and deformable objects.

Contribution

The paper introduces PIEGraph, a novel hybrid approach that integrates analytical physics with equivariant GNNs to enhance data efficiency and physical realism in object dynamics modeling.

Findings

PIEGraph achieves superior accuracy in dynamics prediction over baselines.

The method enables reliable robotic manipulation with limited interaction data.

Effective on both simulation and real-world robotic tasks.

Abstract

Learning data-efficient object dynamics models for robotic manipulation remains challenging, especially for deformable objects. A popular approach is to model objects as sets of 3D particles and learn their motion using graph neural networks. In practice, this is not enough to maintain physical feasibility over long horizons and may require large amounts of interaction data to learn. We introduce PIEGraph, a novel approach to combining analytical physics and data-driven models to capture object dynamics for both rigid and deformable bodies using limited real-world interaction data. PIEGraph consists of two components: (1) a \textbf{P}hysically \textbf{I}nformed particle-based analytical model (implemented as a spring--mass system) to enforce physically feasible motion, and (2) an \textbf{E}quivariant \textbf{Graph} Neural Network with a novel action representation that exploits symmetries in particle interactions to guide the analytical model. We evaluate PIEGraph in simulation and on robot hardware for reorientation and repositioning tasks with ropes, cloth, stuffed animals and rigid objects. We show that our method enables accurate dynamics prediction and reliable downstream robotic manipulation planning, which outperforms state of the art baselines.