Benchmarking Multi-View BEV Object Detection with Mixed Pinhole and Fisheye Cameras

TL;DR

This paper introduces a new benchmark for 3D object detection using mixed pinhole and fisheye cameras, evaluating different architectures and adaptation strategies for robustness against fisheye distortion.

Contribution

It presents the first real-data benchmark combining fisheye and pinhole images, along with systematic evaluation and practical guidelines for robust 3D perception.

Findings

Projection-free architectures are more robust to fisheye distortion.

Distortion-aware view transformation modules improve detection performance.

The benchmark enables systematic evaluation of multi-view BEV detection methods.

Abstract

Modern autonomous driving systems increasingly rely on mixed camera configurations with pinhole and fisheye cameras for full view perception. However, Bird's-Eye View (BEV) 3D object detection models are predominantly designed for pinhole cameras, leading to performance degradation under fisheye distortion. To bridge this gap, we introduce a multi-view BEV detection benchmark with mixed cameras by converting KITTI-360 into nuScenes format. Our study encompasses three adaptations: rectification for zero-shot evaluation and fine-tuning of nuScenes-trained models, distortion-aware view transformation modules (VTMs) via the MEI camera model, and polar coordinate representations to better align with radial distortion. We systematically evaluate three representative BEV architectures, BEVFormer, BEVDet and PETR, across these strategies. We demonstrate that projection-free architectures are inherently more robust and effective against fisheye distortion than other VTMs. This work establishes the first real-data 3D detection benchmark with fisheye and pinhole images and provides systematic adaptation and practical guidelines for designing robust and cost-effective 3D perception systems. The code is available at https://github.com/CesarLiu/FishBEVOD.git.