BAD-Gaussians: Bundle Adjusted Deblur Gaussian Splatting

TL;DR

BAD-Gaussians introduces a novel explicit Gaussian-based method that jointly deblurs images and reconstructs 3D scenes from motion-blurred images with inaccurate camera poses, achieving high-quality, real-time rendering.

Contribution

It presents the first explicit Gaussian representation approach that models motion blur and camera motion jointly for improved 3D scene reconstruction.

Findings

Outperforms previous neural deblurring methods in quality

Enables real-time rendering of motion-blurred scenes

Works effectively on synthetic and real datasets

Abstract

While neural rendering has demonstrated impressive capabilities in 3D scene reconstruction and novel view synthesis, it heavily relies on high-quality sharp images and accurate camera poses. Numerous approaches have been proposed to train Neural Radiance Fields (NeRF) with motion-blurred images, commonly encountered in real-world scenarios such as low-light or long-exposure conditions. However, the implicit representation of NeRF struggles to accurately recover intricate details from severely motion-blurred images and cannot achieve real-time rendering. In contrast, recent advancements in 3D Gaussian Splatting achieve high-quality 3D scene reconstruction and real-time rendering by explicitly optimizing point clouds as Gaussian spheres. In this paper, we introduce a novel approach, named BAD-Gaussians (Bundle Adjusted Deblur Gaussian Splatting), which leverages explicit Gaussian representation and handles severe motion-blurred images with inaccurate camera poses to achieve high-quality scene reconstruction. Our method models the physical image formation process of motion-blurred images and jointly learns the parameters of Gaussians while recovering camera motion trajectories during exposure time. In our experiments, we demonstrate that BAD-Gaussians not only achieves superior rendering quality compared to previous state-of-the-art deblur neural rendering methods on both synthetic and real datasets but also enables real-time rendering capabilities. Our project page and source code is available at https://lingzhezhao.github.io/BAD-Gaussians/