AdaptiGraph: Material-Adaptive Graph-Based Neural Dynamics for Robotic Manipulation

TL;DR

AdaptiGraph is a novel graph neural network-based model that predicts and adapts to the dynamics of various deformable materials with unknown properties, enabling more accurate robotic manipulation without retraining.

Contribution

Introduces a unified, material-conditioned graph neural dynamics model with online few-shot adaptation for diverse deformable objects in robotics.

Findings

Superior prediction accuracy over non-adaptive models

Effective adaptation to new materials during deployment

Demonstrated manipulation on diverse real-world deformable objects

Abstract

Predictive models are a crucial component of many robotic systems. Yet, constructing accurate predictive models for a variety of deformable objects, especially those with unknown physical properties, remains a significant challenge. This paper introduces AdaptiGraph, a learning-based dynamics modeling approach that enables robots to predict, adapt to, and control a wide array of challenging deformable materials with unknown physical properties. AdaptiGraph leverages the highly flexible graph-based neural dynamics (GBND) framework, which represents material bits as particles and employs a graph neural network (GNN) to predict particle motion. Its key innovation is a unified physical property-conditioned GBND model capable of predicting the motions of diverse materials with varying physical properties without retraining. Upon encountering new materials during online deployment, AdaptiGraph utilizes a physical property optimization process for a few-shot adaptation of the model, enhancing its fit to the observed interaction data. The adapted models can precisely simulate the dynamics and predict the motion of various deformable materials, such as ropes, granular media, rigid boxes, and cloth, while adapting to different physical properties, including stiffness, granular size, and center of pressure. On prediction and manipulation tasks involving a diverse set of real-world deformable objects, our method exhibits superior prediction accuracy and task proficiency over non-material-conditioned and non-adaptive models. The project page is available at https://robopil.github.io/adaptigraph/ .