SemOD: Semantic Enabled Object Detection Network under Various Weather Conditions

TL;DR

This paper presents SemOD, a semantic-enabled object detection network designed to improve autonomous driving perception across various weather conditions by leveraging semantic information for image enhancement and detection.

Contribution

Introduces a novel semantic-enabled architecture combining image refinement and object detection, enhancing performance across diverse weather scenarios.

Findings

Achieves up to 8.80% increase in mAP over existing methods.

Effectively uses semantics for image filling and object boundary understanding.

Demonstrates robustness of the approach across multiple weather datasets.

Abstract

In the field of autonomous driving, camera-based perception models are mostly trained on clear weather data. Models that focus on addressing specific weather challenges are unable to adapt to various weather changes and primarily prioritize their weather removal characteristics. Our study introduces a semantic-enabled network for object detection in diverse weather conditions. In our analysis, semantics information can enable the model to generate plausible content for missing areas, understand object boundaries, and preserve visual coherency and realism across both filled-in and existing portions of the image, which are conducive to image transformation and object recognition. Specific in implementation, our architecture consists of a Preprocessing Unit (PPU) and a Detection Unit (DTU), where the PPU utilizes a U-shaped net enriched by semantics to refine degraded images, and the DTU integrates this semantic information for object detection using a modified YOLO network. Our method pioneers the use of semantic data for all-weather transformations, resulting in an increase between 1.47\% to 8.80\% in mAP compared to existing methods across benchmark datasets of different weather. This highlights the potency of semantics in image enhancement and object detection, offering a comprehensive approach to improving object detection performance. Code will be available at https://github.com/EnisZuo/SemOD.