Superpoint Gaussian Splatting for Real-Time High-Fidelity Dynamic Scene Reconstruction

TL;DR

The paper introduces Superpoint Gaussian Splatting (SP-GS), a novel real-time rendering framework for dynamic scenes that achieves high fidelity and manipulation capabilities by extending 3D Gaussian splatting with superpoints.

Contribution

The paper presents a new explicit 3D Gaussian-based method with superpoint clustering for real-time high-quality dynamic scene reconstruction, outperforming existing NeRF-based approaches.

Findings

State-of-the-art visual quality in dynamic scene rendering

Real-time rendering at high resolutions

Effective manipulation of scene components

Abstract

Rendering novel view images in dynamic scenes is a crucial yet challenging task. Current methods mainly utilize NeRF-based methods to represent the static scene and an additional time-variant MLP to model scene deformations, resulting in relatively low rendering quality as well as slow inference speed. To tackle these challenges, we propose a novel framework named Superpoint Gaussian Splatting (SP-GS). Specifically, our framework first employs explicit 3D Gaussians to reconstruct the scene and then clusters Gaussians with similar properties (e.g., rotation, translation, and location) into superpoints. Empowered by these superpoints, our method manages to extend 3D Gaussian splatting to dynamic scenes with only a slight increase in computational expense. Apart from achieving state-of-the-art visual quality and real-time rendering under high resolutions, the superpoint representation provides a stronger manipulation capability. Extensive experiments demonstrate the practicality and effectiveness of our approach on both synthetic and real-world datasets. Please see our project page at https://dnvtmf.github.io/SP_GS.github.io.