Deblurring Neural Radiance Fields with Event-driven Bundle Adjustment

TL;DR

This paper introduces EBAD-NeRF, a novel method that uses event-driven data to jointly optimize camera poses and NeRF parameters, effectively deblurring images and improving 3D reconstruction quality in motion-blurred scenes.

Contribution

The paper presents a new approach combining event data with RGB images to enhance NeRF deblurring and pose estimation, outperforming previous methods.

Findings

EBAD-NeRF accurately estimates camera trajectories during exposure.

The method produces sharper 3D reconstructions than prior approaches.

Experimental results validate effectiveness on synthetic and real data.

Abstract

Neural Radiance Fields (NeRF) achieves impressive 3D representation learning and novel view synthesis results with high-quality multi-view images as input. However, motion blur in images often occurs in low-light and high-speed motion scenes, which significantly degrades the reconstruction quality of NeRF. Previous deblurring NeRF methods struggle to estimate pose and lighting changes during the exposure time, making them unable to accurately model the motion blur. The bio-inspired event camera measuring intensity changes with high temporal resolution makes up this information deficiency. In this paper, we propose Event-driven Bundle Adjustment for Deblurring Neural Radiance Fields (EBAD-NeRF) to jointly optimize the learnable poses and NeRF parameters by leveraging the hybrid event-RGB data. An intensity-change-metric event loss and a photo-metric blur loss are introduced to strengthen the explicit modeling of camera motion blur. Experiments on both synthetic and real-captured data demonstrate that EBAD-NeRF can obtain accurate camera trajectory during the exposure time and learn a sharper 3D representations compared to prior works.