Vision-based Large-scale 3D Semantic Mapping for Autonomous Driving Applications

TL;DR

This paper introduces a stereo camera-based pipeline for large-scale 3D semantic mapping in autonomous driving, integrating visual odometry, global optimization, and semantic labeling, validated on extensive datasets.

Contribution

It presents a novel pipeline combining visual odometry, GNSS integration, and semantic labeling with a temporal voting scheme for improved accuracy.

Findings

Effective semantic labeling with a voting scheme

Successful large-scale mapping over 8000 km

Validated on KITTI-360 dataset

Abstract

In this paper, we present a complete pipeline for 3D semantic mapping solely based on a stereo camera system. The pipeline comprises a direct sparse visual odometry front-end as well as a back-end for global optimization including GNSS integration, and semantic 3D point cloud labeling. We propose a simple but effective temporal voting scheme which improves the quality and consistency of the 3D point labels. Qualitative and quantitative evaluations of our pipeline are performed on the KITTI-360 dataset. The results show the effectiveness of our proposed voting scheme and the capability of our pipeline for efficient large-scale 3D semantic mapping. The large-scale mapping capabilities of our pipeline is furthermore demonstrated by presenting a very large-scale semantic map covering 8000 km of roads generated from data collected by a fleet of vehicles.