BeNeRF: Neural Radiance Fields from a Single Blurry Image and Event Stream

TL;DR

BeNeRF introduces a method to reconstruct neural radiance fields from a single blurry image and event stream, enabling high-quality rendering and camera motion recovery without pre-computed poses.

Contribution

The paper presents a novel approach to recover NeRF from a single blurry image and event stream, jointly learning scene representation and camera motion without relying on external pose estimation.

Findings

Successfully reconstructs view-consistent sharp images from blurry inputs.

Effectively recovers camera motion using cubic B-Spline interpolation.

Demonstrates high-quality rendering on synthetic and real datasets.

Abstract

Neural implicit representation of visual scenes has attracted a lot of attention in recent research of computer vision and graphics. Most prior methods focus on how to reconstruct 3D scene representation from a set of images. In this work, we demonstrate the possibility to recover the neural radiance fields (NeRF) from a single blurry image and its corresponding event stream. We model the camera motion with a cubic B-Spline in SE(3) space. Both the blurry image and the brightness change within a time interval, can then be synthesized from the 3D scene representation given the 6-DoF poses interpolated from the cubic B-Spline. Our method can jointly learn both the implicit neural scene representation and recover the camera motion by minimizing the differences between the synthesized data and the real measurements without pre-computed camera poses from COLMAP. We evaluate the proposed method with both synthetic and real datasets. The experimental results demonstrate that we are able to render view-consistent latent sharp images from the learned NeRF and bring a blurry image alive in high quality. Code and data are available at https://github.com/wu-cvgl/BeNeRF.