Deep Joint Demosaicing and High Dynamic Range Imaging within a Single Shot

TL;DR

This paper introduces a deep learning framework that jointly performs demosaicing and high dynamic range imaging from a single shot with spatially varying exposure, effectively handling over- and under-exposed pixels.

Contribution

It proposes a novel end-to-end deep learning method that integrates spatially varying convolution and exposure-guidance for single-shot HDRI from Bayer images.

Findings

Outperforms state-of-the-art methods in HDRI quality

Effectively handles over- and under-exposed pixels

Avoids cumulative errors in processing pipeline

Abstract

Spatially varying exposure (SVE) is a promising choice for high-dynamic-range (HDR) imaging (HDRI). The SVE-based HDRI, which is called single-shot HDRI, is an efficient solution to avoid ghosting artifacts. However, it is very challenging to restore a full-resolution HDR image from a real-world image with SVE because: a) only one-third of pixels with varying exposures are captured by camera in a Bayer pattern, b) some of the captured pixels are over- and under-exposed. For the former challenge, a spatially varying convolution (SVC) is designed to process the Bayer images carried with varying exposures. For the latter one, an exposure-guidance method is proposed against the interference from over- and under-exposed pixels. Finally, a joint demosaicing and HDRI deep learning framework is formalized to include the two novel components and to realize an end-to-end single-shot HDRI. Experiments indicate that the proposed end-to-end framework avoids the problem of cumulative errors and surpasses the related state-of-the-art methods.