HDR-GAN: HDR Image Reconstruction from Multi-Exposed LDR Images with Large Motions

TL;DR

HDR-GAN is a novel GAN-based method that effectively reconstructs high dynamic range images from multi-exposed LDR images with large motions, addressing misalignment and missing content issues.

Contribution

This work introduces the first GAN-based approach for fusing multi-exposed LDR images for HDR reconstruction, with novel modules for motion alignment and artifact elimination.

Findings

Achieves state-of-the-art HDR reconstruction performance.

Effectively handles large object motions and missing content.

Produces faithful HDR images with minimal artifacts.

Abstract

Synthesizing high dynamic range (HDR) images from multiple low-dynamic range (LDR) exposures in dynamic scenes is challenging. There are two major problems caused by the large motions of foreground objects. One is the severe misalignment among the LDR images. The other is the missing content due to the over-/under-saturated regions caused by the moving objects, which may not be easily compensated for by the multiple LDR exposures. Thus, it requires the HDR generation model to be able to properly fuse the LDR images and restore the missing details without introducing artifacts. To address these two problems, we propose in this paper a novel GAN-based model, HDR-GAN, for synthesizing HDR images from multi-exposed LDR images. To our best knowledge, this work is the first GAN-based approach for fusing multi-exposed LDR images for HDR reconstruction. By incorporating adversarial learning, our method is able to produce faithful information in the regions with missing content. In addition, we also propose a novel generator network, with a reference-based residual merging block for aligning large object motions in the feature domain, and a deep HDR supervision scheme for eliminating artifacts of the reconstructed HDR images. Experimental results demonstrate that our model achieves state-of-the-art reconstruction performance over the prior HDR methods on diverse scenes.