BeSplat: Gaussian Splatting from a Single Blurry Image and Event Stream

TL;DR

BeSplat introduces a novel method to recover sharp radiance fields from a single blurry image and event stream, enabling view-consistent rendering and accurate camera motion estimation, addressing a highly ill-posed problem with temporal data integration.

Contribution

This work is the first to recover Gaussian Splatting-based radiance fields from a single blurry image and event stream, jointly learning scene representation and camera motion.

Findings

Successfully reconstructs sharp view-consistent images from synthetic and real data.

Effectively estimates camera trajectory using Bezier SE(3) formulation.

Demonstrates superior performance over existing methods in handling motion blur and temporal information.

Abstract

Novel view synthesis has been greatly enhanced by the development of radiance field methods. The introduction of 3D Gaussian Splatting (3DGS) has effectively addressed key challenges, such as long training times and slow rendering speeds, typically associated with Neural Radiance Fields (NeRF), while maintaining high-quality reconstructions. In this work (BeSplat), we demonstrate the recovery of sharp radiance field (Gaussian splats) from a single motion-blurred image and its corresponding event stream. Our method jointly learns the scene representation via Gaussian Splatting and recovers the camera motion through Bezier SE(3) formulation effectively, minimizing discrepancies between synthesized and real-world measurements of both blurry image and corresponding event stream. We evaluate our approach on both synthetic and real datasets, showcasing its ability to render view-consistent, sharp images from the learned radiance field and the estimated camera trajectory. To the best of our knowledge, ours is the first work to address this highly challenging ill-posed problem in a Gaussian Splatting framework with the effective incorporation of temporal information captured using the event stream.