Disentangling and Vectorization: A 3D Visual Perception Approach for Autonomous Driving Based on Surround-View Fisheye Cameras

TL;DR

This paper introduces a novel 3D perception method for autonomous vehicles using surround-view fisheye cameras, breaking down detection into sub-tasks and employing multidimensional vectors to improve accuracy and real-time performance.

Contribution

It proposes a new 3D detection framework that avoids large-scale 3D data requirements by decomposing the task and utilizing multidimensional vectors for better information integration.

Findings

Achieves state-of-the-art accuracy on fisheye images

Operates in real-time for practical deployment

Effectively decomposes 3D detection into sub-tasks

Abstract

The 3D visual perception for vehicles with the surround-view fisheye camera system is a critical and challenging task for low-cost urban autonomous driving. While existing monocular 3D object detection methods perform not well enough on the fisheye images for mass production, partly due to the lack of 3D datasets of such images. In this paper, we manage to overcome and avoid the difficulty of acquiring the large scale of accurate 3D labeled truth data, by breaking down the 3D object detection task into some sub-tasks, such as vehicle's contact point detection, type classification, re-identification and unit assembling, etc. Particularly, we propose the concept of Multidimensional Vector to include the utilizable information generated in different dimensions and stages, instead of the descriptive approach for the bird's eye view (BEV) or a cube of eight points. The experiments of real fisheye images demonstrate that our solution achieves state-of-the-art accuracy while being real-time in practice.