Evaluating the Impact of Weather-Induced Sensor Occlusion on BEVFusion for 3D Object Detection

TL;DR

This paper investigates how weather-induced sensor occlusions affect 3D object detection in autonomous vehicles using BEVFusion, revealing the model's reliance on LiDAR and the significant performance drops under occlusion conditions.

Contribution

The study provides a detailed analysis of occlusion impacts on BEVFusion architecture, highlighting the need for occlusion-aware methods in sensor fusion for autonomous driving.

Findings

Camera occlusion causes a 41.3% drop in camera-only detection performance.

Heavy LiDAR occlusion reduces detection accuracy by 47.3%.

Fused detection is more resilient to camera occlusion than LiDAR occlusion.

Abstract

Accurate 3D object detection is essential for automated vehicles to navigate safely in complex real-world environments. Bird's Eye View (BEV) representations, which project multi-sensor data into a top-down spatial format, have emerged as a powerful approach for robust perception. Although BEV-based fusion architectures have demonstrated strong performance through multimodal integration, the effects of sensor occlusions, caused by environmental conditions such as fog, haze, or physical obstructions, on 3D detection accuracy remain underexplored. In this work, we investigate the impact of occlusions on both camera and Light Detection and Ranging (LiDAR) outputs using the BEVFusion architecture, evaluated on the nuScenes dataset. Detection performance is measured using mean Average Precision (mAP) and the nuScenes Detection Score (NDS). Our results show that moderate camera occlusions lead to a 41.3% drop in mAP (from 35.6% to 20.9%) when detection is based only on the camera. On the other hand, LiDAR sharply drops in performance only under heavy occlusion, with mAP falling by 47.3% (from 64.7% to 34.1%), with a severe impact on long-range detection. In fused settings, the effect depends on which sensor is occluded: occluding the camera leads to a minor 4.1% drop (from 68.5% to 65.7%), while occluding LiDAR results in a larger 26.8% drop (to 50.1%), revealing the model's stronger reliance on LiDAR for the task of 3D object detection. Our results highlight the need for future research into occlusion-aware evaluation methods and improved sensor fusion techniques that can maintain detection accuracy in the presence of partial sensor failure or degradation due to adverse environmental conditions.