Clustered Error Correction with Grouped 4D Gaussian Splatting

TL;DR

This paper introduces a novel error correction and grouping method for 4D Gaussian Splatting, significantly enhancing dynamic scene reconstruction accuracy and temporal consistency in neural rendering.

Contribution

The paper proposes a new error correction technique and grouped 4D Gaussian Splatting to improve dynamic scene reconstruction and consistency in neural rendering.

Findings

Achieves state-of-the-art perceptual rendering quality.

Improves PSNR by 0.39dB on Technicolor dataset.

Enhances alignment between splats and dynamic objects.

Abstract

Existing 4D Gaussian Splatting (4DGS) methods struggle to accurately reconstruct dynamic scenes, often failing to resolve ambiguous pixel correspondences and inadequate densification in dynamic regions. We address these issues by introducing a novel method composed of two key components: (1) Elliptical Error Clustering and Error Correcting Splat Addition that pinpoints dynamic areas to improve and initialize fitting splats, and (2) Grouped 4D Gaussian Splatting that improves consistency of mapping between splats and represented dynamic objects. Specifically, we classify rendering errors into missing-color and occlusion types, then apply targeted corrections via backprojection or foreground splitting guided by cross-view color consistency. Evaluations on Neural 3D Video and Technicolor datasets demonstrate that our approach significantly improves temporal consistency and achieves state-of-the-art perceptual rendering quality, improving 0.39dB of PSNR on the Technicolor Light Field dataset. Our visualization shows improved alignment between splats and dynamic objects, and the error correction method's capability to identify errors and properly initialize new splats. Our implementation details and source code are available at https://github.com/tho-kn/cem-4dgs.