Predicting Physical Object Properties from Video

TL;DR

This paper introduces a flexible method combining physics simulation and correction estimation to accurately predict physical object properties from video, applicable with various physics engines and estimators.

Contribution

It presents a novel, generic framework that integrates physics simulation with correction estimation, improving accuracy and convergence in property prediction from video data.

Findings

Faster convergence with the learned method.

Robustness in simulated 2D scenarios.

Effective use of both hyperparameter optimization and neural networks.

Abstract

We present a novel approach to estimating physical properties of objects from video. Our approach consists of a physics engine and a correction estimator. Starting from the initial observed state, object behavior is simulated forward in time. Based on the simulated and observed behavior, the correction estimator then determines refined physical parameters for each object. The method can be iterated for increased precision. Our approach is generic, as it allows for the use of an arbitrary - not necessarily differentiable - physics engine and correction estimator. For the latter, we evaluate both gradient-free hyperparameter optimization and a deep convolutional neural network. We demonstrate faster and more robust convergence of the learned method in several simulated 2D scenarios focusing on bin situations.