Learning sRGB-to-Raw-RGB De-rendering with Content-Aware Metadata

TL;DR

This paper introduces a content-aware learning approach for de-rendering sRGB images back to raw-RGB format, improving accuracy over traditional methods by jointly learning sampling and reconstruction strategies.

Contribution

It proposes a novel joint learning framework for sampling and reconstruction in raw de-rendering, with an online fine-tuning strategy for enhanced results.

Findings

Learned sampling adapts to image content for better raw reconstruction.

Joint learning outperforms existing simple sampling and global mapping methods.

Online fine-tuning further improves de-rendering accuracy.

Abstract

Most camera images are rendered and saved in the standard RGB (sRGB) format by the camera's hardware. Due to the in-camera photo-finishing routines, nonlinear sRGB images are undesirable for computer vision tasks that assume a direct relationship between pixel values and scene radiance. For such applications, linear raw-RGB sensor images are preferred. Saving images in their raw-RGB format is still uncommon due to the large storage requirement and lack of support by many imaging applications. Several "raw reconstruction" methods have been proposed that utilize specialized metadata sampled from the raw-RGB image at capture time and embedded in the sRGB image. This metadata is used to parameterize a mapping function to de-render the sRGB image back to its original raw-RGB format when needed. Existing raw reconstruction methods rely on simple sampling strategies and global mapping to perform the de-rendering. This paper shows how to improve the de-rendering results by jointly learning sampling and reconstruction. Our experiments show that our learned sampling can adapt to the image content to produce better raw reconstructions than existing methods. We also describe an online fine-tuning strategy for the reconstruction network to improve results further.