W-Net: Two-stage U-Net with misaligned data for raw-to-RGB mapping

TL;DR

This paper introduces W-Net, a two-stage U-Net architecture with a specialized loss function for raw-to-RGB image mapping, effectively handling misaligned data and improving image quality metrics.

Contribution

The paper proposes a novel two-stage U-Net model and a robust loss function tailored for misaligned raw-RGB data, advancing raw-to-RGB mapping techniques.

Findings

Achieved highest PSNR and SSIM scores in the AIM 2019 challenge.

Ensemble of networks with different loss functions improves performance.

Outperforms existing methods on the Zurich Raw-to-RGB dataset.

Abstract

Recent research on learning a mapping between raw Bayer images and RGB images has progressed with the development of deep convolutional neural networks. A challenging data set namely the Zurich Raw-to-RGB data set (ZRR) has been released in the AIM 2019 raw-to-RGB mapping challenge. In ZRR, input raw and target RGB images are captured by two different cameras and thus not perfectly aligned. Moreover, camera metadata such as white balance gains and color correction matrix are not provided, which makes the challenge more difficult. In this paper, we explore an effective network structure and a loss function to address these issues. We exploit a two-stage U-Net architecture and also introduce a loss function that is less variant to alignment and more sensitive to color differences. In addition, we show an ensemble of networks trained with different loss functions can bring a significant performance gain. We demonstrate the superiority of our method by achieving the highest score in terms of both the peak signal-to-noise ratio and the structural similarity and obtaining the second-best mean-opinion-score in the challenge.