BEVal: A Cross-dataset Evaluation Study of BEV Segmentation Models for Autonomous Driving

TL;DR

This study evaluates the generalization of BEV segmentation models across multiple datasets and sensor setups, highlighting the importance of multi-dataset training for robustness in autonomous driving.

Contribution

It provides a comprehensive cross-dataset evaluation of BEV segmentation models and demonstrates the benefits of multi-dataset training for improved generalization.

Findings

Cross-dataset evaluation reveals models' limited generalization across different environments.

Multi-dataset training enhances BEV segmentation performance over single-dataset training.

Sensor type influences the models' ability to generalize to diverse conditions.

Abstract

Current research in semantic bird's-eye view segmentation for autonomous driving focuses solely on optimizing neural network models using a single dataset, typically nuScenes. This practice leads to the development of highly specialized models that may fail when faced with different environments or sensor setups, a problem known as domain shift. In this paper, we conduct a comprehensive cross-dataset evaluation of state-of-the-art BEV segmentation models to assess their performance across different training and testing datasets and setups, as well as different semantic categories. We investigate the influence of different sensors, such as cameras and LiDAR, on the models' ability to generalize to diverse conditions and scenarios. Additionally, we conduct multi-dataset training experiments that improve models' BEV segmentation performance compared to single-dataset training. Our work addresses the gap in evaluating BEV segmentation models under cross-dataset validation. And our findings underscore the importance of enhancing model generalizability and adaptability to ensure more robust and reliable BEV segmentation approaches for autonomous driving applications. The code for this paper available at https://github.com/manueldiaz96/beval .