MASIV: Toward Material-Agnostic System Identification from Videos

TL;DR

MASIV is a novel vision-based framework that enables material-agnostic system identification from videos by using learnable neural models and dense geometric guidance, overcoming prior limitations of predefined material priors.

Contribution

It introduces the first neural, material-agnostic system identification framework that infers object dynamics without scene-specific material priors, utilizing dense geometric guidance for stability.

Findings

Achieves state-of-the-art geometric accuracy

Provides high-quality rendering results

Demonstrates strong generalization to unseen materials

Abstract

System identification from videos aims to recover object geometry and governing physical laws. Existing methods integrate differentiable rendering with simulation but rely on predefined material priors, limiting their ability to handle unknown ones. We introduce MASIV, the first vision-based framework for material-agnostic system identification. Unlike existing approaches that depend on hand-crafted constitutive laws, MASIV employs learnable neural constitutive models, inferring object dynamics without assuming a scene-specific material prior. However, the absence of full particle state information imposes unique challenges, leading to unstable optimization and physically implausible behaviors. To address this, we introduce dense geometric guidance by reconstructing continuum particle trajectories, providing temporally rich motion constraints beyond sparse visual cues. Comprehensive experiments show that MASIV achieves state-of-the-art performance in geometric accuracy, rendering quality, and generalization ability.