DENSER: 3D Gaussians Splatting for Scene Reconstruction of Dynamic Urban Environments

TL;DR

DENSER introduces a novel framework using 3D Gaussian splatting and wavelet-based dynamic SH estimation to improve scene reconstruction of dynamic urban environments, capturing detailed object appearance and shape.

Contribution

The paper proposes DENSER, a new method that enhances dynamic object modeling and shape densification for more accurate scene reconstruction.

Findings

Outperforms state-of-the-art methods on KITTI dataset

Improves dynamic object appearance representation

Speeds up model training convergence

Abstract

This paper presents DENSER, an efficient and effective approach leveraging 3D Gaussian splatting (3DGS) for the reconstruction of dynamic urban environments. While several methods for photorealistic scene representations, both implicitly using neural radiance fields (NeRF) and explicitly using 3DGS have shown promising results in scene reconstruction of relatively complex dynamic scenes, modeling the dynamic appearance of foreground objects tend to be challenging, limiting the applicability of these methods to capture subtleties and details of the scenes, especially far dynamic objects. To this end, we propose DENSER, a framework that significantly enhances the representation of dynamic objects and accurately models the appearance of dynamic objects in the driving scene. Instead of directly using Spherical Harmonics (SH) to model the appearance of dynamic objects, we introduce and integrate a new method aiming at dynamically estimating SH bases using wavelets, resulting in better representation of dynamic objects appearance in both space and time. Besides object appearance, DENSER enhances object shape representation through densification of its point cloud across multiple scene frames, resulting in faster convergence of model training. Extensive evaluations on KITTI dataset show that the proposed approach significantly outperforms state-of-the-art methods by a wide margin. Source codes and models will be uploaded to this repository https://github.com/sntubix/denser