DynSUP: Dynamic Gaussian Splatting from An Unposed Image Pair

TL;DR

DynSUP introduces a novel method for dynamic scene 3D reconstruction from just two unposed images, combining object-level bundle adjustment and learnable Gaussian transformations for high-fidelity view synthesis.

Contribution

It presents the first approach to model dynamic scenes with only two images and no prior poses, using a two-view bundle adjustment and SE(3) Gaussian transformations.

Findings

Outperforms state-of-the-art static scene methods on synthetic and real data.

Achieves high-fidelity novel view synthesis with accurate motion and temporal consistency.

Effectively models dynamic objects with minimal input data.

Abstract

Recent advances in 3D Gaussian Splatting have shown promising results. Existing methods typically assume static scenes and/or multiple images with prior poses. Dynamics, sparse views, and unknown poses significantly increase the problem complexity due to insufficient geometric constraints. To overcome this challenge, we propose a method that can use only two images without prior poses to fit Gaussians in dynamic environments. To achieve this, we introduce two technical contributions. First, we propose an object-level two-view bundle adjustment. This strategy decomposes dynamic scenes into piece-wise rigid components, and jointly estimates the camera pose and motions of dynamic objects. Second, we design an SE(3) field-driven Gaussian training method. It enables fine-grained motion modeling through learnable per-Gaussian transformations. Our method leads to high-fidelity novel view synthesis of dynamic scenes while accurately preserving temporal consistency and object motion. Experiments on both synthetic and real-world datasets demonstrate that our method significantly outperforms state-of-the-art approaches designed for the cases of static environments, multiple images, and/or known poses. Our project page is available at https://colin-de.github.io/DynSUP/.