Real-Time Dense Mapping for Self-driving Vehicles using Fisheye Cameras

TL;DR

This paper introduces a real-time dense mapping algorithm for self-driving vehicles using fisheye cameras, enabling efficient 3D scene perception in large-scale environments at 15 Hz.

Contribution

It is the first to develop a real-time dense mapping method specifically optimized for fisheye cameras in autonomous driving contexts.

Findings

Works effectively in complex urban environments

Operates at approximately 15 Hz on in-vehicle PCs

Accurately recovers depths using multi-resolution plane-sweeping stereo

Abstract

We present a real-time dense geometric mapping algorithm for large-scale environments. Unlike existing methods which use pinhole cameras, our implementation is based on fisheye cameras which have larger field of view and benefit some other tasks including Visual-Inertial Odometry, localization and object detection around vehicles. Our algorithm runs on in-vehicle PCs at 15 Hz approximately, enabling vision-only 3D scene perception for self-driving vehicles. For each synchronized set of images captured by multiple cameras, we first compute a depth map for a reference camera using plane-sweeping stereo. To maintain both accuracy and efficiency, while accounting for the fact that fisheye images have a rather low resolution, we recover the depths using multiple image resolutions. We adopt the fast object detection framework YOLOv3 to remove potentially dynamic objects. At the end of the pipeline, we fuse the fisheye depth images into the truncated signed distance function (TSDF) volume to obtain a 3D map. We evaluate our method on large-scale urban datasets, and results show that our method works well even in complex environments.