NeuSpring: Neural Spring Fields for Reconstruction and Simulation of Deformable Objects from Videos

TL;DR

NeuSpring introduces a neural spring field approach that models deformable objects from videos, improving both reconstruction accuracy and future physical prediction by incorporating intrinsic physical properties and topology modeling.

Contribution

It presents a novel neural spring field method with topology modeling and a neural network for physical property learning, enhancing deformable object simulation from videos.

Findings

20% improvement in Chamfer distance for reconstruction

25% improvement in Chamfer distance for future prediction

Superior performance over existing methods in real-world datasets

Abstract

In this paper, we aim to create physical digital twins of deformable objects under interaction. Existing methods focus more on the physical learning of current state modeling, but generalize worse to future prediction. This is because existing methods ignore the intrinsic physical properties of deformable objects, resulting in the limited physical learning in the current state modeling. To address this, we present NeuSpring, a neural spring field for the reconstruction and simulation of deformable objects from videos. Built upon spring-mass models for realistic physical simulation, our method consists of two major innovations: 1) a piecewise topology solution that efficiently models multi-region spring connection topologies using zero-order optimization, which considers the material heterogeneity of real-world objects. 2) a neural spring field that represents spring physical properties across different frames using a canonical coordinate-based neural network, which effectively leverages the spatial associativity of springs for physical learning. Experiments on real-world datasets demonstrate that our NeuSping achieves superior reconstruction and simulation performance for current state modeling and future prediction, with Chamfer distance improved by 20% and 25%, respectively.