AW-MoE: All-Weather Mixture of Experts for Robust Multi-Modal 3D Object Detection

TL;DR

AW-MoE introduces a weather-aware mixture of experts framework for robust multi-modal 3D object detection, significantly improving performance under adverse weather conditions by leveraging image-guided routing and dual-modal augmentation.

Contribution

The paper proposes AW-MoE, integrating Mixture of Experts with weather-aware routing and dual-modal augmentation, to enhance robustness in multi-modal 3D detection across diverse weather scenarios.

Findings

Achieves approximately 15% improvement in adverse-weather detection performance.

Effectively handles data discrepancies across weather conditions with minimal inference overhead.

Enhances baseline detectors with significant performance gains in real-world tests.

Abstract

Robust 3D object detection under adverse weather conditions is crucial for autonomous driving. However, most existing methods simply combine all weather samples for training while overlooking data distribution discrepancies across different weather scenarios, leading to performance conflicts. To address this issue, we introduce AW-MoE, the framework that innovatively integrates Mixture of Experts (MoE) into weather-robust multi-modal 3D object detection approaches. AW-MoE incorporates Image-guided Weather-aware Routing (IWR), which leverages the superior discriminability of image features across weather conditions and their invariance to scene variations for precise weather classification. Based on this accurate classification, IWR selects the top-K most relevant Weather-Specific Experts (WSE) that handle data discrepancies, ensuring optimal detection under all weather conditions. Additionally, we propose a Unified Dual-Modal Augmentation (UDMA) for synchronous LiDAR and 4D Radar dual-modal data augmentation while preserving the realism of scenes. Extensive experiments on the real-world dataset demonstrate that AW-MoE achieves ~ 15% improvement in adverse-weather performance over state-of-the-art methods, while incurring negligible inference overhead. Moreover, integrating AW-MoE into established baseline detectors yields performance improvements surpassing current state-of-the-art methods. These results show the effectiveness and strong scalability of our AW-MoE. We will release the code publicly at https://github.com/windlinsherlock/AW-MoE.