Semantic Aware Diffusion Inverse Tone Mapping

TL;DR

This paper introduces a novel semantic-aware diffusion inpainting method for inverse tone mapping, effectively recovering details in clipped regions of SDR images to produce high-quality HDR images, outperforming existing methods.

Contribution

The paper presents a new approach combining semantic graph-guided diffusion inpainting with a principled HDR lifting formulation, advancing inverse tone mapping techniques.

Findings

Superior performance on multiple datasets

Outperforms state-of-the-art in objective metrics

Achieves better visual fidelity in HDR images

Abstract

The range of real-world scene luminance is larger than the capture capability of many digital camera sensors which leads to details being lost in captured images, most typically in bright regions. Inverse tone mapping attempts to boost these captured Standard Dynamic Range (SDR) images back to High Dynamic Range (HDR) by creating a mapping that linearizes the well exposed values from the SDR image, and provides a luminance boost to the clipped content. However, in most cases, the details in the clipped regions cannot be recovered or estimated. In this paper, we present a novel inverse tone mapping approach for mapping SDR images to HDR that generates lost details in clipped regions through a semantic-aware diffusion based inpainting approach. Our method proposes two major contributions - first, we propose to use a semantic graph to guide SDR diffusion based inpainting in masked regions in a saturated image. Second, drawing inspiration from traditional HDR imaging and bracketing methods, we propose a principled formulation to lift the SDR inpainted regions to HDR that is compatible with generative inpainting methods. Results show that our method demonstrates superior performance across different datasets on objective metrics, and subjective experiments show that the proposed method matches (and in most cases outperforms) state-of-art inverse tone mapping operators in terms of objective metrics and outperforms them for visual fidelity.