3D Gaussian Representations with Motion Trajectory Field for Dynamic Scene Reconstruction

TL;DR

This paper introduces a novel method combining 3D Gaussian Splatting with a motion trajectory field to improve dynamic scene reconstruction and motion prediction from monocular videos, achieving state-of-the-art results.

Contribution

It presents a new approach that decouples dynamic objects from static backgrounds using a motion trajectory field with shared bases, enhancing motion modeling in dynamic scenes.

Findings

Achieves state-of-the-art novel-view synthesis results.

Effectively reconstructs complex object motions.

Handles dynamic scenes with improved accuracy.

Abstract

This paper addresses the challenge of novel-view synthesis and motion reconstruction of dynamic scenes from monocular video, which is critical for many robotic applications. Although Neural Radiance Fields (NeRF) and 3D Gaussian Splatting (3DGS) have demonstrated remarkable success in rendering static scenes, extending them to reconstruct dynamic scenes remains challenging. In this work, we introduce a novel approach that combines 3DGS with a motion trajectory field, enabling precise handling of complex object motions and achieving physically plausible motion trajectories. By decoupling dynamic objects from static background, our method compactly optimizes the motion trajectory field. The approach incorporates time-invariant motion coefficients and shared motion trajectory bases to capture intricate motion patterns while minimizing optimization complexity. Extensive experiments demonstrate that our approach achieves state-of-the-art results in both novel-view synthesis and motion trajectory recovery from monocular video, advancing the capabilities of dynamic scene reconstruction.