Lightness Modulated Deep Inverse Tone Mapping

TL;DR

This paper introduces a deep learning-based inverse tone mapping method that uses a hierarchical synthesis network and lightness adaptive modulation to improve HDR reconstruction, especially in over-exposed regions.

Contribution

It proposes a novel hierarchical synthesis network combined with a lightness adaptive modulation network that effectively utilizes lightness priors for better HDR image recovery.

Findings

Outperforms state-of-the-art methods in HDR reconstruction quality.

Demonstrates effectiveness through quantitative and visual comparisons.

Provides ablation studies and activation map visualizations for deeper understanding.

Abstract

Single-image HDR reconstruction or inverse tone mapping (iTM) is a challenging task. In particular, recovering information in over-exposed regions is extremely difficult because details in such regions are almost completely lost. In this paper, we present a deep learning based iTM method that takes advantage of the feature extraction and mapping power of deep convolutional neural networks (CNNs) and uses a lightness prior to modulate the CNN to better exploit observations in the surrounding areas of the over-exposed regions to enhance the quality of HDR image reconstruction. Specifically, we introduce a Hierarchical Synthesis Network (HiSN) for inferring a HDR image from a LDR input and a Lightness Adpative Modulation Network (LAMN) to incorporate the the lightness prior knowledge in the inferring process. The HiSN hierarchically synthesizes the high-brightness component and the low-brightness component of the HDR image whilst the LAMN uses a lightness adaptive mask that separates detail-less saturated bright pixels from well-exposed lower light pixels to enable HiSN to better infer the missing information, particularly in the difficult over-exposed detail-less areas. We present experimental results to demonstrate the effectiveness of the new technique based on quantitative measures and visual comparisons. In addition, we present ablation studies of HiSN and visualization of the activation maps inside LAMN to help gain a deeper understanding of the internal working of the new iTM algorithm and explain why it can achieve much improved performance over state-of-the-art algorithms.