DGNS: Deformable Gaussian Splatting and Dynamic Neural Surface for Monocular Dynamic 3D Reconstruction

TL;DR

DGNS is a hybrid framework that combines deformable Gaussian splatting and dynamic neural surfaces to improve monocular dynamic 3D reconstruction and novel-view synthesis, achieving state-of-the-art results.

Contribution

The paper introduces DGNS, a novel hybrid approach integrating deformable Gaussian splatting with dynamic neural surfaces for improved dynamic scene reconstruction.

Findings

Achieves state-of-the-art 3D reconstruction performance.

Provides competitive results in novel-view synthesis.

Introduces a depth-filtering approach for better depth supervision.

Abstract

Dynamic scene reconstruction from monocular video is essential for real-world applications. We introduce DGNS, a hybrid framework integrating \underline{D}eformable \underline{G}aussian Splatting and Dynamic \underline{N}eural \underline{S}urfaces, effectively addressing dynamic novel-view synthesis and 3D geometry reconstruction simultaneously. During training, depth maps generated by the deformable Gaussian splatting module guide the ray sampling for faster processing and provide depth supervision within the dynamic neural surface module to improve geometry reconstruction. Conversely, the dynamic neural surface directs the distribution of Gaussian primitives around the surface, enhancing rendering quality. In addition, we propose a depth-filtering approach to further refine depth supervision. Extensive experiments conducted on public datasets demonstrate that DGNS achieves state-of-the-art performance in 3D reconstruction, along with competitive results in novel-view synthesis.