SMASH: Physics-guided Reconstruction of Collisions from Videos

TL;DR

SMASH is a novel method that reconstructs physically plausible 3D collision sequences from raw videos by leveraging rigid body collision laws, facilitating easier editing and synthesis of collision events.

Contribution

This work introduces a physics-guided approach to recover collision parameters and 3D trajectories directly from videos, improving the realism and controllability of synthetic collision generation.

Findings

Accurately reconstructs complex real-world collisions

Effectively estimates collision parameters from videos

Enables editing and synthesis of new collision sequences

Abstract

Collision sequences are commonly used in games and entertainment to add drama and excitement. Authoring even two body collisions in the real world can be difficult, as one has to get timing and the object trajectories to be correctly synchronized. After tedious trial-and-error iterations, when objects can actually be made to collide, then they are difficult to capture in 3D. In contrast, synthetically generating plausible collisions is difficult as it requires adjusting different collision parameters (e.g., object mass ratio, coefficient of restitution, etc.) and appropriate initial parameters. We present SMASH to directly read off appropriate collision parameters directly from raw input video recordings. Technically we enable this by utilizing laws of rigid body collision to regularize the problem of lifting 2D trajectories to a physically valid 3D reconstruction of the collision. The reconstructed sequences can then be modified and combined to easily author novel and plausible collisions. We evaluate our system on a range of synthetic scenes and demonstrate the effectiveness of our method by accurately reconstructing several complex real world collision events.