Exploring Surround-View Fisheye Camera 3D Object Detection

TL;DR

This paper investigates 3D object detection using surround-view fisheye cameras, proposing new methods that incorporate fisheye geometry and introducing a new dataset, resulting in improved detection accuracy.

Contribution

The paper introduces two fisheye-specific 3D detection methods and a new dataset, addressing the performance drop of traditional detectors on fisheye imagery.

Findings

Fisheye-specific models outperform baseline methods by up to 6.2%

Developed FisheyeBEVDet and FisheyePETR for fisheye 3D detection

Released Fisheye3DOD dataset for benchmarking

Abstract

In this work, we explore the technical feasibility of implementing end-to-end 3D object detection (3DOD) with surround-view fisheye camera system. Specifically, we first investigate the performance drop incurred when transferring classic pinhole-based 3D object detectors to fisheye imagery. To mitigate this, we then develop two methods that incorporate the unique geometry of fisheye images into mainstream detection frameworks: one based on the bird's-eye-view (BEV) paradigm, named FisheyeBEVDet, and the other on the query-based paradigm, named FisheyePETR. Both methods adopt spherical spatial representations to effectively capture fisheye geometry. In light of the lack of dedicated evaluation benchmarks, we release Fisheye3DOD, a new open dataset synthesized using CARLA and featuring both standard pinhole and fisheye camera arrays. Experiments on Fisheye3DOD show that our fisheye-compatible modeling improves accuracy by up to 6.2% over baseline methods.