Robust Gaussian Splatting

TL;DR

This paper enhances 3D Gaussian Splatting by modeling motion and defocus blur, and correcting color inconsistencies, leading to more robust reconstructions from handheld captures with state-of-the-art results.

Contribution

It introduces a unified approach to handle motion blur, defocus, and color inconsistencies within 3D Gaussian Splatting, improving robustness for practical applications.

Findings

Achieved state-of-the-art results on Scannet++ and Deblur-NeRF datasets.

Improved robustness of 3D reconstructions from handheld phone captures.

Maintained training efficiency and rendering speed despite added robustness mechanisms.

Abstract

In this paper, we address common error sources for 3D Gaussian Splatting (3DGS) including blur, imperfect camera poses, and color inconsistencies, with the goal of improving its robustness for practical applications like reconstructions from handheld phone captures. Our main contribution involves modeling motion blur as a Gaussian distribution over camera poses, allowing us to address both camera pose refinement and motion blur correction in a unified way. Additionally, we propose mechanisms for defocus blur compensation and for addressing color in-consistencies caused by ambient light, shadows, or due to camera-related factors like varying white balancing settings. Our proposed solutions integrate in a seamless way with the 3DGS formulation while maintaining its benefits in terms of training efficiency and rendering speed. We experimentally validate our contributions on relevant benchmark datasets including Scannet++ and Deblur-NeRF, obtaining state-of-the-art results and thus consistent improvements over relevant baselines.