Learning RAW-to-sRGB Mappings with Inaccurately Aligned Supervision

TL;DR

This paper introduces a joint learning approach for RAW-to-sRGB mapping that effectively handles misaligned supervision by combining a global color mapping module and optical flow for image alignment, improving accuracy and robustness.

Contribution

The paper proposes a novel joint learning model that integrates a global color mapping and optical flow to address misalignment issues in RAW-to-sRGB translation tasks.

Findings

Outperforms state-of-the-art methods on ZRR and SR-RAW datasets.

Achieves better quantitative and qualitative results with a lightweight backbone.

Detaches modules after training, ensuring no extra inference cost.

Abstract

Learning RAW-to-sRGB mapping has drawn increasing attention in recent years, wherein an input raw image is trained to imitate the target sRGB image captured by another camera. However, the severe color inconsistency makes it very challenging to generate well-aligned training pairs of input raw and target sRGB images. While learning with inaccurately aligned supervision is prone to causing pixel shift and producing blurry results. In this paper, we circumvent such issue by presenting a joint learning model for image alignment and RAW-to-sRGB mapping. To diminish the effect of color inconsistency in image alignment, we introduce to use a global color mapping (GCM) module to generate an initial sRGB image given the input raw image, which can keep the spatial location of the pixels unchanged, and the target sRGB image is utilized to guide GCM for converting the color towards it. Then a pre-trained optical flow estimation network (e.g., PWC-Net) is deployed to warp the target sRGB image to align with the GCM output. To alleviate the effect of inaccurately aligned supervision, the warped target sRGB image is leveraged to learn RAW-to-sRGB mapping. When training is done, the GCM module and optical flow network can be detached, thereby bringing no extra computation cost for inference. Experiments show that our method performs favorably against state-of-the-arts on ZRR and SR-RAW datasets. With our joint learning model, a light-weight backbone can achieve better quantitative and qualitative performance on ZRR dataset. Codes are available at https://github.com/cszhilu1998/RAW-to-sRGB.