GeMS: Efficient Gaussian Splatting for Extreme Motion Blur

TL;DR

GeMS is a novel framework that reconstructs 3D scenes directly from severely motion-blurred images by integrating deep learning-based pose estimation, probabilistic scene initialization, and event-based deblurring, achieving state-of-the-art results.

Contribution

This work introduces GeMS, the first 3D Gaussian Splatting method capable of handling extreme motion blur directly from blurred images, combining novel pose estimation, probabilistic initialization, and event-based refinement.

Findings

Achieves state-of-the-art 3D reconstruction from blurred images.

Effectively integrates event-based deblurring with 3D Gaussian Splatting.

Demonstrates robustness on synthetic and real-world datasets.

Abstract

We introduce GeMS, a framework for 3D Gaussian Splatting (3DGS) designed to handle severely motion-blurred images. State-of-the-art deblurring methods for extreme blur, such as ExBluRF, as well as Gaussian Splatting-based approaches like Deblur-GS, typically assume access to sharp images for camera pose estimation and point cloud generation, an unrealistic assumption. Methods relying on COLMAP initialization, such as BAD-Gaussians, also fail due to unreliable feature correspondences under severe blur. To address these challenges, we propose GeMS, a 3DGS framework that reconstructs scenes directly from extremely blurred images. GeMS integrates: (1) VGGSfM, a deep learning-based Structure-from-Motion pipeline that estimates poses and generates point clouds directly from blurred inputs; (2) 3DGS-MCMC, which enables robust scene initialization by treating Gaussians as samples from a probability distribution, eliminating heuristic densification and pruning; and (3) joint optimization of camera trajectories and Gaussian parameters for stable reconstruction. While this pipeline produces strong results, inaccuracies may remain when all inputs are severely blurred. To mitigate this, we propose GeMS-E, which integrates a progressive refinement step using events: (4) Event-based Double Integral (EDI) deblurring restores sharper images that are then fed into GeMS, improving pose estimation, point cloud generation, and overall reconstruction. Both GeMS and GeMS-E achieve state-of-the-art performance on synthetic and real-world datasets. To our knowledge, this is the first framework to address extreme motion blur within 3DGS directly from severely blurred inputs.