Few-Shot Neural Differentiable Simulator: Real-to-Sim Rigid-Contact Modeling

TL;DR

This paper introduces a few-shot real-to-sim approach that combines analytical physics models with graph neural networks to create highly accurate, differentiable simulators for robotic contact dynamics, requiring minimal real-world data.

Contribution

The paper presents a novel method that calibrates analytical simulators with limited real data and employs a mesh-based GNN for fully differentiable rigid-body dynamics modeling.

Findings

Outperforms baseline simulators in replicating real trajectories

Enables efficient gradient-based policy optimization in complex scenarios

Requires only small amounts of real-world data for calibration

Abstract

Accurate physics simulation is essential for robotic learning and control, yet analytical simulators often fail to capture complex contact dynamics, while learning-based simulators typically require large amounts of costly real-world data. To bridge this gap, we propose a few-shot real-to-sim approach that combines the physical consistency of analytical formulations with the representational capacity of graph neural network (GNN)-based models. Using only a small amount of real-world data, our method calibrates analytical simulators to generate large-scale synthetic datasets that capture diverse contact interactions. On this foundation, we introduce a mesh-based GNN that implicitly models rigid-body forward dynamics and derive surrogate gradients for collision detection, achieving full differentiability. Experimental results demonstrate that our approach enables learning-based simulators to outperform differentiable baselines in replicating real-world trajectories. In addition, the differentiable design supports gradient-based optimization, which we validate through simulation-based policy learning in multi-object interaction scenarios. Extensive experiments show that our framework not only improves simulation fidelity with minimal supervision but also increases the efficiency of policy learning. Taken together, these findings suggest that differentiable simulation with few-shot real-world grounding provides a powerful direction for advancing future robotic manipulation and control.