Modelling the Scene Dependent Imaging in Cameras with a Deep Neural Network

TL;DR

This paper introduces a deep learning framework that models scene-dependent camera image processing, enabling accurate RAW image recovery from JPEGs and improving image deblurring by capturing local and global scene context.

Contribution

It presents a data-driven approach to model scene-dependent, locally varying camera imaging pipelines under auto mode, surpassing previous deterministic models.

Findings

Accurately models camera imaging pipelines in both directions

Improves image deblurring performance

Handles scene-dependent and local variations

Abstract

We present a novel deep learning framework that models the scene dependent image processing inside cameras. Often called as the radiometric calibration, the process of recovering RAW images from processed images (JPEG format in the sRGB color space) is essential for many computer vision tasks that rely on physically accurate radiance values. All previous works rely on the deterministic imaging model where the color transformation stays the same regardless of the scene and thus they can only be applied for images taken under the manual mode. In this paper, we propose a data-driven approach to learn the scene dependent and locally varying image processing inside cameras under the automode. Our method incorporates both the global and the local scene context into pixel-wise features via multi-scale pyramid of learnable histogram layers. The results show that we can model the imaging pipeline of different cameras that operate under the automode accurately in both directions (from RAW to sRGB, from sRGB to RAW) and we show how we can apply our method to improve the performance of image deblurring.