LRC-WeatherNet: LiDAR, RADAR, and Camera Fusion Network for Real-time Weather-type Classification in Autonomous Driving

TL;DR

LRC-WeatherNet is a multi-sensor fusion network that combines LiDAR, RADAR, and camera data to classify weather conditions in real-time, improving robustness and accuracy for autonomous vehicles in adverse weather.

Contribution

This paper introduces the first multi-modal fusion framework for real-time weather classification using LiDAR, RADAR, and camera data in autonomous driving.

Findings

Outperforms unimodal baselines in adverse weather conditions

Achieves high classification accuracy and computational efficiency

Successfully integrates three sensor modalities for robust weather detection

Abstract

Autonomous vehicles face major perception and navigation challenges in adverse weather such as rain, fog, and snow, which degrade the performance of LiDAR, RADAR, and RGB camera sensors. While each sensor type offers unique strengths, such as RADAR robustness in poor visibility and LiDAR precision in clear conditions, they also suffer distinct limitations when exposed to environmental obstructions. This study proposes LRC-WeatherNet, a novel multi-sensor fusion framework that integrates LiDAR, RADAR, and camera data for real-time classification of weather conditions. By employing both early fusion using a unified Bird's Eye View representation and mid-level gated fusion of modality-specific feature maps, our approach adapts to the varying reliability of each sensor under changing weather. Evaluated on the extensive MSU-4S dataset covering nine weather types, LRC-WeatherNet achieves superior classification performance and computational efficiency, significantly outperforming unimodal baselines in adverse conditions. This work is the first to combine all three modalities for robust, real-time weather classification in autonomous driving. We release our trained models and source code in https://github.com/nouralhudaalbashir/LRC-WeatherNet.