Self-supervised HDR Imaging from Motion and Exposure Cues

TL;DR

This paper introduces a self-supervised method for HDR imaging that uses internal image statistics to generate pseudo-labels, enabling HDR reconstruction without ground-truth labels and performing competitively with supervised approaches.

Contribution

The work presents a novel self-supervised HDR estimation technique leveraging internal image statistics and synthetic augmentations, reducing reliance on complex ground-truth data.

Findings

Self-supervised HDR models achieve performance comparable to supervised methods.

The approach outperforms previous unsupervised HDR techniques.

High-quality HDR results are obtained without HDR ground-truth labels.

Abstract

Recent High Dynamic Range (HDR) techniques extend the capabilities of current cameras where scenes with a wide range of illumination can not be accurately captured with a single low-dynamic-range (LDR) image. This is generally accomplished by capturing several LDR images with varying exposure values whose information is then incorporated into a merged HDR image. While such approaches work well for static scenes, dynamic scenes pose several challenges, mostly related to the difficulty of finding reliable pixel correspondences. Data-driven approaches tackle the problem by learning an end-to-end mapping with paired LDR-HDR training data, but in practice generating such HDR ground-truth labels for dynamic scenes is time-consuming and requires complex procedures that assume control of certain dynamic elements of the scene (e.g. actor pose) and repeatable lighting conditions (stop-motion capturing). In this work, we propose a novel self-supervised approach for learnable HDR estimation that alleviates the need for HDR ground-truth labels. We propose to leverage the internal statistics of LDR images to create HDR pseudo-labels. We separately exploit static and well-exposed parts of the input images, which in conjunction with synthetic illumination clipping and motion augmentation provide high quality training examples. Experimental results show that the HDR models trained using our proposed self-supervision approach achieve performance competitive with those trained under full supervision, and are to a large extent superior to previous methods that equally do not require any supervision.