RAWtoBit: A Fully End-to-end Camera ISP Network

TL;DR

RAWtoBit is an end-to-end deep learning model that integrates image compression into the camera ISP pipeline, achieving superior rate-distortion performance.

Contribution

We introduce RAWtoBit, a novel fully end-to-end optimized camera ISP network that combines image processing and compression, enhanced by a knowledge distillation scheme.

Findings

Significantly outperforms existing methods in rate-distortion trade-off.

Effectively performs both image processing and compression tasks simultaneously.

Knowledge distillation improves model performance.

Abstract

Image compression is an essential and last processing unit in the camera image signal processing (ISP) pipeline. While many studies have been made to replace the conventional ISP pipeline with a single end-to-end optimized deep learning model, image compression is barely considered as a part of the model. In this paper, we investigate the designing of a fully end-to-end optimized camera ISP incorporating image compression. To this end, we propose RAWtoBit network (RBN) that can effectively perform both tasks simultaneously. RBN is further improved with a novel knowledge distillation scheme by introducing two teacher networks specialized in each task. Extensive experiments demonstrate that our proposed method significantly outperforms alternative approaches in terms of rate-distortion trade-off.