SyMetric: Measuring the Quality of Learnt Hamiltonian Dynamics Inferred from Vision

TL;DR

This paper introduces SyMetric, a new evaluation metric for Hamiltonian dynamics models learned from images, which better reflects the models' ability to capture underlying physics than traditional reconstruction error.

Contribution

The paper develops SyMetric, a discriminative measure based on Hamiltonian properties, and demonstrates its effectiveness in improving and evaluating latent dynamics models like HGN.

Findings

SyMetric effectively distinguishes models that accurately learn Hamiltonian dynamics.

The improved HGN++ model discovers interpretable phase space and maintains stable, high-quality long-term rollouts.

HGN++ outperforms previous models on multiple datasets in capturing physical dynamics.

Abstract

A recently proposed class of models attempts to learn latent dynamics from high-dimensional observations, like images, using priors informed by Hamiltonian mechanics. While these models have important potential applications in areas like robotics or autonomous driving, there is currently no good way to evaluate their performance: existing methods primarily rely on image reconstruction quality, which does not always reflect the quality of the learnt latent dynamics. In this work, we empirically highlight the problems with the existing measures and develop a set of new measures, including a binary indicator of whether the underlying Hamiltonian dynamics have been faithfully captured, which we call Symplecticity Metric or SyMetric. Our measures take advantage of the known properties of Hamiltonian dynamics and are more discriminative of the model's ability to capture the underlying dynamics than reconstruction error. Using SyMetric, we identify a set of architectural choices that significantly improve the performance of a previously proposed model for inferring latent dynamics from pixels, the Hamiltonian Generative Network (HGN). Unlike the original HGN, the new HGN++ is able to discover an interpretable phase space with physically meaningful latents on some datasets. Furthermore, it is stable for significantly longer rollouts on a diverse range of 13 datasets, producing rollouts of essentially infinite length both forward and backwards in time with no degradation in quality on a subset of the datasets.