Video-Driven Graph Network-Based Simulators

TL;DR

This paper introduces a video-driven approach that infers physical properties from short videos and uses graph networks to simulate system trajectories, reducing the need for explicit physical parameters.

Contribution

It proposes a novel method that learns physical representations directly from videos and integrates them into graph network simulators, streamlining physics-based animation.

Findings

Video-derived encodings effectively capture physical properties.

Linear relationship observed between encodings and system motion.

Method reduces reliance on detailed physical input.

Abstract

Lifelike visualizations in design, cinematography, and gaming rely on precise physics simulations, typically requiring extensive computational resources and detailed physical input. This paper presents a method that can infer a system's physical properties from a short video, eliminating the need for explicit parameter input, provided it is close to the training condition. The learned representation is then used within a Graph Network-based Simulator to emulate the trajectories of physical systems. We demonstrate that the video-derived encodings effectively capture the physical properties of the system and showcase a linear dependence between some of the encodings and the system's motion.