UniDet-D: A Unified Dynamic Spectral Attention Model for Object Detection under Adverse Weathers

TL;DR

UniDet-D is a unified model that combines object detection and image restoration using dynamic spectral attention to handle diverse adverse weather conditions effectively.

Contribution

It introduces a novel unified framework with spectral attention for robust object detection and restoration across multiple adverse weather scenarios.

Findings

Achieves superior detection accuracy across various weather degradations.

Demonstrates strong generalization to unseen adverse weather conditions.

Outperforms existing methods in robustness and accuracy.

Abstract

Real-world object detection is a challenging task where the captured images/videos often suffer from complex degradations due to various adverse weather conditions such as rain, fog, snow, low-light, etc. Despite extensive prior efforts, most existing methods are designed for one specific type of adverse weather with constraints of poor generalization, under-utilization of visual features while handling various image degradations. Leveraging a theoretical analysis on how critical visual details are lost in adverse-weather images, we design UniDet-D, a unified framework that tackles the challenge of object detection under various adverse weather conditions, and achieves object detection and image restoration within a single network. Specifically, the proposed UniDet-D incorporates a dynamic spectral attention mechanism that adaptively emphasizes informative spectral components while suppressing irrelevant ones, enabling more robust and discriminative feature representation across various degradation types. Extensive experiments show that UniDet-D achieves superior detection accuracy across different types of adverse-weather degradation. Furthermore, UniDet-D demonstrates superior generalization towards unseen adverse weather conditions such as sandstorms and rain-fog mixtures, highlighting its great potential for real-world deployment.