FisheyeDetNet: 360{\deg} Surround view Fisheye Camera based Object Detection System for Autonomous Driving

TL;DR

This paper introduces FisheyeDetNet, a novel object detection system for fisheye cameras in autonomous driving, utilizing specialized bounding box representations to handle radial distortion, and achieves state-of-the-art accuracy on a large global dataset.

Contribution

It proposes new bounding box representations tailored for fisheye distortion and demonstrates their effectiveness with a high-performing detection model on a comprehensive dataset.

Findings

FisheyeDetNet achieves 49.5% mAP on the Valeo dataset.

Polygon-based bounding boxes outperform other representations.

First detailed study on fisheye camera object detection for autonomous driving.

Abstract

Object detection is a mature problem in autonomous driving with pedestrian detection being one of the first deployed algorithms. It has been comprehensively studied in the literature. However, object detection is relatively less explored for fisheye cameras used for surround-view near field sensing. The standard bounding box representation fails in fisheye cameras due to heavy radial distortion, particularly in the periphery. To mitigate this, we explore extending the standard object detection output representation of bounding box. We design rotated bounding boxes, ellipse, generic polygon as polar arc/angle representations and define an instance segmentation mIOU metric to analyze these representations. The proposed model FisheyeDetNet with polygon outperforms others and achieves a mAP score of 49.5 % on Valeo fisheye surround-view dataset for automated driving applications. This dataset has 60K images captured from 4 surround-view cameras across Europe, North America and Asia. To the best of our knowledge, this is the first detailed study on object detection on fisheye cameras for autonomous driving scenarios.