Broadening View Synthesis of Dynamic Scenes from Constrained Monocular Videos

TL;DR

This paper introduces ExpanDyNeRF, a novel monocular NeRF framework that improves dynamic scene view synthesis under large viewpoint changes using Gaussian splatting and a new synthetic dataset.

Contribution

The paper presents ExpanDyNeRF, a new method for dynamic view synthesis with large viewpoint deviations, and introduces the SynDM dataset for training and evaluation.

Findings

ExpanDyNeRF outperforms existing methods on SynDM and real datasets.

The synthetic dataset enables better training with explicit side-view supervision.

The approach achieves more stable and realistic renderings under extreme viewpoint shifts.

Abstract

In dynamic Neural Radiance Fields (NeRF) systems, state-of-the-art novel view synthesis methods often fail under significant viewpoint deviations, producing unstable and unrealistic renderings. To address this, we introduce Expanded Dynamic NeRF (ExpanDyNeRF), a monocular NeRF framework that leverages Gaussian splatting priors and a pseudo-ground-truth generation strategy to enable realistic synthesis under large-angle rotations. ExpanDyNeRF optimizes density and color features to improve scene reconstruction from challenging perspectives. We also present the Synthetic Dynamic Multiview (SynDM) dataset, the first synthetic multiview dataset for dynamic scenes with explicit side-view supervision-created using a custom GTA V-based rendering pipeline. Quantitative and qualitative results on SynDM and real-world datasets demonstrate that ExpanDyNeRF significantly outperforms existing dynamic NeRF methods in rendering fidelity under extreme viewpoint shifts. Further details are provided in the supplementary materials.