Raw or Cooked? Object Detection on RAW Images

TL;DR

This paper explores whether using RAW images instead of processed RGB images improves object detection performance, proposing a learnable ISP operation that optimizes for the detection task.

Contribution

It introduces a novel learnable ISP operation that jointly optimizes image processing parameters for better object detection on RAW images.

Findings

Learnable ISP improves detection accuracy on RAW images.

Empirical validation on PASCALRAW dataset supports hypothesis.

Outperforms previous methods and traditional RGB images.

Abstract

Images fed to a deep neural network have in general undergone several handcrafted image signal processing (ISP) operations, all of which have been optimized to produce visually pleasing images. In this work, we investigate the hypothesis that the intermediate representation of visually pleasing images is sub-optimal for downstream computer vision tasks compared to the RAW image representation. We suggest that the operations of the ISP instead should be optimized towards the end task, by learning the parameters of the operations jointly during training. We extend previous works on this topic and propose a new learnable operation that enables an object detector to achieve superior performance when compared to both previous works and traditional RGB images. In experiments on the open PASCALRAW dataset, we empirically confirm our hypothesis.