Deblur e-NeRF: NeRF from Motion-Blurred Events under High-speed or Low-light Conditions

TL;DR

Deblur e-NeRF introduces a method to reconstruct high-quality neural radiance fields from motion-blurred event camera data, addressing a gap in existing approaches by modeling event motion blur under various conditions.

Contribution

It proposes a physically-accurate pixel bandwidth model for event motion blur and a novel regularization loss, enabling effective NeRF reconstruction from blurred event data.

Findings

Effective reconstruction of NeRFs from blurred events demonstrated on real and simulated data.

The proposed model outperforms existing methods in high-speed and low-light scenarios.

Open-source code and datasets facilitate further research.

Abstract

The stark contrast in the design philosophy of an event camera makes it particularly ideal for operating under high-speed, high dynamic range and low-light conditions, where standard cameras underperform. Nonetheless, event cameras still suffer from some amount of motion blur, especially under these challenging conditions, in contrary to what most think. This is attributed to the limited bandwidth of the event sensor pixel, which is mostly proportional to the light intensity. Thus, to ensure that event cameras can truly excel in such conditions where it has an edge over standard cameras, it is crucial to account for event motion blur in downstream applications, especially reconstruction. However, none of the recent works on reconstructing Neural Radiance Fields (NeRFs) from events, nor event simulators, have considered the full effects of event motion blur. To this end, we propose, Deblur e-NeRF, a novel method to directly and effectively reconstruct blur-minimal NeRFs from motion-blurred events generated under high-speed motion or low-light conditions. The core component of this work is a physically-accurate pixel bandwidth model proposed to account for event motion blur under arbitrary speed and lighting conditions. We also introduce a novel threshold-normalized total variation loss to improve the regularization of large textureless patches. Experiments on real and novel realistically simulated sequences verify our effectiveness. Our code, event simulator and synthetic event dataset will be open-sourced.