Weather-Conditioned Branch Routing for Robust LiDAR-Radar 3D Object Detection

TL;DR

This paper introduces a weather-conditioned branch routing framework for robust 3D object detection using LiDAR and radar, dynamically adjusting modality reliance based on weather conditions to improve performance and interpretability.

Contribution

It reformulates multi-modal perception as a weather-conditioned routing problem, enabling dynamic sensor modality weighting and providing interpretability in adverse weather scenarios.

Findings

Achieves state-of-the-art results on the K-Radar benchmark.

Provides explicit insights into modality reliance shifts across weather conditions.

Introduces a weather-supervised learning strategy with diversity regularization.

Abstract

Robust 3D object detection in adverse weather is highly challenging due to the varying reliability of different sensors. While existing LiDAR-4D radar fusion methods improve robustness, they predominantly rely on fixed or weakly adaptive pipelines, failing to dy-namically adjust modality preferences as environmental conditions change. To bridge this gap, we reformulate multi-modal perception as a weather-conditioned branch routing problem. Instead of computing a single fused output, our framework explicitly maintains three parallel 3D feature streams: a pure LiDAR branch, a pure 4D radar branch, and a condition-gated fusion branch. Guided by a condition token extracted from visual and semantic prompts, a lightweight router dynamically predicts sample-specific weights to softly aggregate these representations. Furthermore, to prevent branch collapse, we introduce a weather-supervised learning strategy with auxiliary classification and diversity regularization to enforce distinct, condition-dependent routing behaviors. Extensive experiments on the K-Radar benchmark demonstrate that our method achieves state-of-the-art performance. Furthermore, it provides explicit and highly interpretable insights into modality preferences, transparently revealing how adaptive routing robustly shifts reliance between LiDAR and 4D radar across diverse adverse-weather scenarios. The source code with be released.