DeepISP: Towards Learning an End-to-End Image Processing Pipeline

TL;DR

DeepISP is an end-to-end deep learning model that replicates and surpasses traditional camera image processing pipelines, improving image quality directly from raw sensor data.

Contribution

It introduces a comprehensive neural network model that learns the entire camera ISP pipeline, including demosaicing, denoising, and color correction, trained on real paired datasets.

Findings

Achieves state-of-the-art PSNR in joint denoising and demosaicing.

Outperforms manufacturer ISP in subjective visual quality.

Demonstrates effective end-to-end learning of camera image processing.

Abstract

We present DeepISP, a full end-to-end deep neural model of the camera image signal processing (ISP) pipeline. Our model learns a mapping from the raw low-light mosaiced image to the final visually compelling image and encompasses low-level tasks such as demosaicing and denoising as well as higher-level tasks such as color correction and image adjustment. The training and evaluation of the pipeline were performed on a dedicated dataset containing pairs of low-light and well-lit images captured by a Samsung S7 smartphone camera in both raw and processed JPEG formats. The proposed solution achieves state-of-the-art performance in objective evaluation of PSNR on the subtask of joint denoising and demosaicing. For the full end-to-end pipeline, it achieves better visual quality compared to the manufacturer ISP, in both a subjective human assessment and when rated by a deep model trained for assessing image quality.