Towards Real World HDRTV Reconstruction: A Data Synthesis-based Approach

TL;DR

This paper introduces a data synthesis method that models real-world SDRTV degradation more accurately for training HDRTV reconstruction networks, improving their generalization to real-world data.

Contribution

A novel learning-based data synthesis approach incorporating tone mapping priors into network structures and loss functions for realistic SDRTV-HDRTV pair generation.

Findings

Networks trained with synthesized data outperform existing methods on real-world datasets.

The proposed synthesis method captures real-world degradation more effectively.

Experimental results show improved generalization to real SDRTVs.

Abstract

Existing deep learning based HDRTV reconstruction methods assume one kind of tone mapping operators (TMOs) as the degradation procedure to synthesize SDRTV-HDRTV pairs for supervised training. In this paper, we argue that, although traditional TMOs exploit efficient dynamic range compression priors, they have several drawbacks on modeling the realistic degradation: information over-preservation, color bias and possible artifacts, making the trained reconstruction networks hard to generalize well to real-world cases. To solve this problem, we propose a learning-based data synthesis approach to learn the properties of real-world SDRTVs by integrating several tone mapping priors into both network structures and loss functions. In specific, we design a conditioned two-stream network with prior tone mapping results as a guidance to synthesize SDRTVs by both global and local transformations. To train the data synthesis network, we form a novel self-supervised content loss to constraint different aspects of the synthesized SDRTVs at regions with different brightness distributions and an adversarial loss to emphasize the details to be more realistic. To validate the effectiveness of our approach, we synthesize SDRTV-HDRTV pairs with our method and use them to train several HDRTV reconstruction networks. Then we collect two inference datasets containing both labeled and unlabeled real-world SDRTVs, respectively. Experimental results demonstrate that, the networks trained with our synthesized data generalize significantly better to these two real-world datasets than existing solutions.