Deep Camera: A Fully Convolutional Neural Network for Image Signal Processing

TL;DR

This paper introduces a fully convolutional neural network that performs the entire image signal processing pipeline in a camera, improving image quality over traditional sequential methods.

Contribution

The paper presents the first end-to-end CNN model capable of handling all steps of image signal processing in a camera.

Findings

CNN outperforms traditional pipelines in image quality

End-to-end training reduces residual errors

System generalizes well across diverse images

Abstract

A conventional camera performs various signal processing steps sequentially to reconstruct an image from a raw Bayer image. When performing these processing in multiple stages the residual error from each stage accumulates in the image and degrades the quality of the final reconstructed image. In this paper, we present a fully convolutional neural network (CNN) to perform defect pixel correction, denoising, white balancing, exposure correction, demosaicing, color transform, and gamma encoding. To our knowledge, this is the first CNN trained end-to-end to perform the entire image signal processing pipeline in a camera. The neural network was trained using a large image database of raw Bayer images. Through extensive experiments, we show that the proposed CNN based image signal processing system performs better than the conventional signal processing pipelines that perform the processing sequentially.