Point Cloud Sequence Encoding for Material-conditioned Graph Network Simulators

TL;DR

PEACH is a novel point cloud sequence encoder that enables material-conditioned graph network simulators to adapt to unseen physical properties during inference, improving zero-shot transfer and prediction accuracy.

Contribution

Introduces PEACH, a spatio-temporal point cloud encoder with auxiliary supervision, for in-context learning of GNSs to handle unseen material parameters.

Findings

PEACH achieves accurate zero-shot sim-to-real transfer on dynamic scenes.

PEACH outperforms mesh-based baselines in prediction accuracy.

PEACH is more practical for real-world deployment than traditional mesh-based methods.

Abstract

Graph Network Simulators (GNSs) have emerged as powerful surrogates for complex physics-based simulation, offering inherent differentiability and orders-of-magnitude speedups over traditional solvers. However, GNSs typically assume access to the underlying material parameters, such as stiffness or viscosity, severely limiting their utility in realistic experimental settings. While recent meta-learning approaches address the parameter dependency by inferring properties from mesh trajectories, reconstructing a mesh from an observed scene is challenging. In this work, we introduce Point Cloud Encoding for Accurate Context Handling (PEACH), a novel framework that applies in-context learning on point clouds to adapt a learned simulator to unseen physical properties during inference. Our approach relies on a novel spatio-temporal point cloud sequence encoder, as well as two forms of auxiliary supervision to help improve simulation fidelity. We demonstrate that PEACH is capable of accurate zero-shot sim-to-real transfer on a challenging, dynamic scene. Experiments on simulation scenes show that PEACH even outperforms mesh-based baselines on prediction accuracy, while being much more practical for real-world deployment.