A Refined 3D Gaussian Representation for High-Quality Dynamic Scene Reconstruction

TL;DR

This paper introduces a refined 3D Gaussian representation that enhances high-quality dynamic scene reconstruction by reducing memory usage and improving rendering speed and quality, using a deformable MLP, hash encoding, and denoising techniques.

Contribution

It proposes a novel dynamic 3D Gaussian model with noise reduction and motion constraints, advancing scene reconstruction efficiency and quality over previous methods.

Findings

Outperforms existing methods in rendering quality and speed

Reduces memory usage significantly

Effective noise removal and motion consistency enforcement

Abstract

In recent years, Neural Radiance Fields (NeRF) has revolutionized three-dimensional (3D) reconstruction with its implicit representation. Building upon NeRF, 3D Gaussian Splatting (3D-GS) has departed from the implicit representation of neural networks and instead directly represents scenes as point clouds with Gaussian-shaped distributions. While this shift has notably elevated the rendering quality and speed of radiance fields but inevitably led to a significant increase in memory usage. Additionally, effectively rendering dynamic scenes in 3D-GS has emerged as a pressing challenge. To address these concerns, this paper purposes a refined 3D Gaussian representation for high-quality dynamic scene reconstruction. Firstly, we use a deformable multi-layer perceptron (MLP) network to capture the dynamic offset of Gaussian points and express the color features of points through hash encoding and a tiny MLP to reduce storage requirements. Subsequently, we introduce a learnable denoising mask coupled with denoising loss to eliminate noise points from the scene, thereby further compressing 3D Gaussian model. Finally, motion noise of points is mitigated through static constraints and motion consistency constraints. Experimental results demonstrate that our method surpasses existing approaches in rendering quality and speed, while significantly reducing the memory usage associated with 3D-GS, making it highly suitable for various tasks such as novel view synthesis, and dynamic mapping.