Contact-Aware Neural Dynamics

TL;DR

This paper introduces a contact-aware neural dynamics model that aligns simulation with real-world contact data, improving prediction accuracy and policy transfer in contact-rich robotic tasks.

Contribution

It presents an implicit sim-to-real alignment framework that leverages tactile contact information to refine simulator dynamics with real-world data.

Findings

Improved state prediction accuracy in contact-rich tasks

Effective policy performance prediction and refinement

Scalable data-driven approach to sim-to-real transfer

Abstract

High-fidelity physics simulation is essential for scalable robotic learning, but the sim-to-real gap persists, especially for tasks involving complex, dynamic, and discontinuous interactions like physical contacts. Explicit system identification, which tunes explicit simulator parameters, is often insufficient to align the intricate, high-dimensional, and state-dependent dynamics of the real world. To overcome this, we propose an implicit sim-to-real alignment framework that learns to directly align the simulator's dynamics with contact information. Our method treats the off-the-shelf simulator as a base prior and learns a contact-aware neural dynamics model to refine simulated states using real-world observations. We show that using tactile contact information from robotic hands can effectively model the non-smooth discontinuities inherent in contact-rich tasks, resulting in a neural dynamics model grounded by real-world data. We demonstrate that this learned forward dynamics model improves state prediction accuracy and can be effectively used to predict policy performance and refine policies trained purely in standard simulators, offering a scalable, data-driven approach to sim-to-real alignment.