Towards RAW Object Detection in Diverse Conditions

TL;DR

This paper introduces the AODRaw dataset with high-resolution RAW images for object detection under diverse conditions, highlighting the domain gap between RAW and sRGB and proposing knowledge distillation to improve RAW detection performance.

Contribution

The paper presents a new RAW image dataset and benchmark for object detection, and explores RAW pre-training with knowledge distillation to enhance detection in challenging conditions.

Findings

RAW pre-training outperforms sRGB pre-training in diverse conditions

Knowledge distillation from sRGB models improves RAW detection

RAW-based models perform better under adverse lighting and weather

Abstract

Existing object detection methods often consider sRGB input, which was compressed from RAW data using ISP originally designed for visualization. However, such compression might lose crucial information for detection, especially under complex light and weather conditions. We introduce the AODRaw dataset, which offers 7,785 high-resolution real RAW images with 135,601 annotated instances spanning 62 categories, capturing a broad range of indoor and outdoor scenes under 9 distinct light and weather conditions. Based on AODRaw that supports RAW and sRGB object detection, we provide a comprehensive benchmark for evaluating current detection methods. We find that sRGB pre-training constrains the potential of RAW object detection due to the domain gap between sRGB and RAW, prompting us to directly pre-train on the RAW domain. However, it is harder for RAW pre-training to learn rich representations than sRGB pre-training due to the camera noise. To assist RAW pre-training, we distill the knowledge from an off-the-shelf model pre-trained on the sRGB domain. As a result, we achieve substantial improvements under diverse and adverse conditions without relying on extra pre-processing modules. Code and dataset are available at https://github.com/lzyhha/AODRaw.