Thermodynamics-informed neural networks for physically realistic mixed reality

TL;DR

This paper introduces a thermodynamics-informed neural network approach for realistic, real-time physics simulation of deformable objects in mixed reality, ensuring thermodynamic consistency and enhancing user experience.

Contribution

The method combines deep learning with thermodynamic principles to simulate physically realistic interactions in mixed reality environments.

Findings

Successfully simulated deformable objects interacting with virtual and physical entities.

Ensured thermodynamic consistency in neural network predictions.

Enhanced realism and responsiveness in mixed reality physics simulations.

Abstract

The imminent impact of immersive technologies in society urges for active research in real-time and interactive physics simulation for virtual worlds to be realistic. In this context, realistic means to be compliant to the laws of physics. In this paper we present a method for computing the dynamic response of (possibly non-linear and dissipative) deformable objects induced by real-time user interactions in mixed reality using deep learning. The graph-based architecture of the method ensures the thermodynamic consistency of the predictions, whereas the visualization pipeline allows a natural and realistic user experience. Two examples of virtual solids interacting with virtual or physical solids in mixed reality scenarios are provided to prove the performance of the method.