Learning Physics From Video: Unsupervised Physical Parameter Estimation for Continuous Dynamical Systems

TL;DR

This paper introduces an unsupervised approach to estimate physical parameters of continuous dynamical systems from videos, eliminating the need for large labeled datasets and frame prediction, and demonstrating effectiveness on synthetic and real-world data.

Contribution

It proposes a novel unsupervised method that uses a KL-divergence-based loss in the latent space, applicable to various dynamical systems and robust to initialization.

Findings

Outperforms existing methods on synthetic data

Successfully applied to real-world videos with diverse dynamical systems

Reduces training time and model complexity

Abstract

Extracting physical dynamical system parameters from recorded observations is key in natural science. Current methods for automatic parameter estimation from video train supervised deep networks on large datasets. Such datasets require labels, which are difficult to acquire. While some unsupervised techniques--which depend on frame prediction--exist, they suffer from long training times, initialization instabilities, only consider motion-based dynamical systems, and are evaluated mainly on synthetic data. In this work, we propose an unsupervised method to estimate the physical parameters of known, continuous governing equations from single videos suitable for different dynamical systems beyond motion and robust to initialization. Moreover, we remove the need for frame prediction by implementing a KL-divergence-based loss function in the latent space, which avoids convergence to trivial solutions and reduces model size and compute. We first evaluate our model on synthetic data, as commonly done. After which, we take the field closer to reality by recording Delfys75: our own real-world dataset of 75 videos for five different types of dynamical systems to evaluate our method and others. Our method compares favorably to others. %, yet, and real-world video datasets and demonstrate improved parameter estimation accuracy compared to existing methods. Code and data are available online:https://github.com/Alejandro-neuro/Learning_physics_from_video.