Benchmarking and Improving Bird's Eye View Perception Robustness in Autonomous Driving

TL;DR

This paper introduces RoboBEV, a comprehensive benchmark suite for evaluating the robustness of bird's eye view perception models in autonomous driving under diverse conditions, and proposes strategies to enhance their resilience.

Contribution

We present RoboBEV, the first extensive benchmark for BEV perception robustness, and demonstrate effective methods like pre-training and temporal information to improve model resilience.

Findings

Model performance correlates with robustness to out-of-distribution data.

Pre-training and depth-free BEV transformations enhance robustness.

Temporal information significantly improves model resilience.

Abstract

Recent advancements in bird's eye view (BEV) representations have shown remarkable promise for in-vehicle 3D perception. However, while these methods have achieved impressive results on standard benchmarks, their robustness in varied conditions remains insufficiently assessed. In this study, we present RoboBEV, an extensive benchmark suite designed to evaluate the resilience of BEV algorithms. This suite incorporates a diverse set of camera corruption types, each examined over three severity levels. Our benchmarks also consider the impact of complete sensor failures that occur when using multi-modal models. Through RoboBEV, we assess 33 state-of-the-art BEV-based perception models spanning tasks like detection, map segmentation, depth estimation, and occupancy prediction. Our analyses reveal a noticeable correlation between the model's performance on in-distribution datasets and its resilience to out-of-distribution challenges. Our experimental results also underline the efficacy of strategies like pre-training and depth-free BEV transformations in enhancing robustness against out-of-distribution data. Furthermore, we observe that leveraging extensive temporal information significantly improves the model's robustness. Based on our observations, we design an effective robustness enhancement strategy based on the CLIP model. The insights from this study pave the way for the development of future BEV models that seamlessly combine accuracy with real-world robustness.