Regularizing Dynamic Radiance Fields with Kinematic Fields

TL;DR

This paper introduces a method that combines kinematic fields with dynamic radiance fields to improve 3D reconstruction of moving scenes from monocular videos, ensuring physically valid motion modeling without requiring motion ground truth.

Contribution

The novel integration of kinematic fields with dynamic radiance fields, along with physics-based regularizers, enables more accurate and physically consistent motion reconstruction from monocular videos.

Findings

Outperforms state-of-the-art methods in real-world monocular videos

Ensures physical validity of predicted motion through regularizers

Effectively captures complex motion patterns in dynamic scenes

Abstract

This paper presents a novel approach for reconstructing dynamic radiance fields from monocular videos. We integrate kinematics with dynamic radiance fields, bridging the gap between the sparse nature of monocular videos and the real-world physics. Our method introduces the kinematic field, capturing motion through kinematic quantities: velocity, acceleration, and jerk. The kinematic field is jointly learned with the dynamic radiance field by minimizing the photometric loss without motion ground truth. We further augment our method with physics-driven regularizers grounded in kinematics. We propose physics-driven regularizers that ensure the physical validity of predicted kinematic quantities, including advective acceleration and jerk. Additionally, we control the motion trajectory based on rigidity equations formed with the predicted kinematic quantities. In experiments, our method outperforms the state-of-the-arts by capturing physical motion patterns within challenging real-world monocular videos.