Event-assisted 12-stop HDR Imaging of Dynamic Scene

TL;DR

This paper introduces a 12-stop HDR imaging method for dynamic scenes using a dual-camera system with an event camera and RGB camera, improving alignment and reducing artifacts in challenging lighting conditions.

Contribution

It presents a novel dual-camera HDR approach with event-based alignment, a diffusion-based fusion module, and the first dataset for 12-stop HDR with event signals, advancing dynamic scene HDR imaging.

Findings

Achieves state-of-the-art 12-stop HDR imaging in dynamic scenes.

Effectively reduces ghosting artifacts caused by motion.

Validates approach on both simulated and real-world data.

Abstract

High dynamic range (HDR) imaging is a crucial task in computational photography, which captures details across diverse lighting conditions. Traditional HDR fusion methods face limitations in dynamic scenes with extreme exposure differences, as aligning low dynamic range (LDR) frames becomes challenging due to motion and brightness variation. In this work, we propose a novel 12-stop HDR imaging approach for dynamic scenes, leveraging a dual-camera system with an event camera and an RGB camera. The event camera provides temporally dense, high dynamic range signals that improve alignment between LDR frames with large exposure differences, reducing ghosting artifacts caused by motion. Also, a real-world finetuning strategy is proposed to increase the generalization of alignment module on real-world events. Additionally, we introduce a diffusion-based fusion module that incorporates image priors from pre-trained diffusion models to address artifacts in high-contrast regions and minimize errors from the alignment process. To support this work, we developed the ESHDR dataset, the first dataset for 12-stop HDR imaging with synchronized event signals, and validated our approach on both simulated and real-world data. Extensive experiments demonstrate that our method achieves state-of-the-art performance, successfully extending HDR imaging to 12 stops in dynamic scenes.