MonoWAD: Weather-Adaptive Diffusion Model for Robust Monocular 3D Object Detection

TL;DR

MonoWAD is a novel weather-adaptive diffusion model that significantly improves monocular 3D object detection robustness across diverse weather conditions, including foggy scenarios, by incorporating weather-aware feature enhancement.

Contribution

The paper introduces MonoWAD, a weather-robust monocular 3D detector utilizing a weather codebook and adaptive diffusion model for improved performance in adverse weather.

Findings

Achieves robust 3D detection in foggy and clear weather.

Outperforms existing methods under various weather conditions.

Demonstrates effectiveness of weather-adaptive feature enhancement.

Abstract

Monocular 3D object detection is an important challenging task in autonomous driving. Existing methods mainly focus on performing 3D detection in ideal weather conditions, characterized by scenarios with clear and optimal visibility. However, the challenge of autonomous driving requires the ability to handle changes in weather conditions, such as foggy weather, not just clear weather. We introduce MonoWAD, a novel weather-robust monocular 3D object detector with a weather-adaptive diffusion model. It contains two components: (1) the weather codebook to memorize the knowledge of the clear weather and generate a weather-reference feature for any input, and (2) the weather-adaptive diffusion model to enhance the feature representation of the input feature by incorporating a weather-reference feature. This serves an attention role in indicating how much improvement is needed for the input feature according to the weather conditions. To achieve this goal, we introduce a weather-adaptive enhancement loss to enhance the feature representation under both clear and foggy weather conditions. Extensive experiments under various weather conditions demonstrate that MonoWAD achieves weather-robust monocular 3D object detection. The code and dataset are released at https://github.com/VisualAIKHU/MonoWAD.