An Online Semantic Mapping System for Extending and Enhancing Visual SLAM

TL;DR

This paper introduces a real-time semantic mapping system that enhances visual SLAM by integrating 2D-3D object detection, semantic constraints, and robust data association, significantly improving pose accuracy and robustness.

Contribution

The novel system combines real-time semantic mapping with pose correction in SLAM, using tracklets and an uncertainty-based association scheme for improved accuracy and efficiency.

Findings

Achieves real-time processing with 65 ms per iteration.

Improves SLAM pose estimation accuracy by up to 68%.

Easily integrable as a modular ROS package.

Abstract

We present a real-time semantic mapping approach for mobile vision systems with a 2D to 3D object detection pipeline and rapid data association for generated landmarks. Besides the semantic map enrichment the associated detections are further introduced as semantic constraints into a simultaneous localization and mapping (SLAM) system for pose correction purposes. This way, we are able generate additional meaningful information that allows to achieve higher-level tasks, while simultaneously leveraging the view-invariance of object detections to improve the accuracy and the robustness of the odometry estimation. We propose tracklets of locally associated object observations to handle ambiguous and false predictions and an uncertainty-based greedy association scheme for an accelerated processing time. Our system reaches real-time capabilities with an average iteration duration of 65~ms and is able to improve the pose estimation of a state-of-the-art SLAM by up to 68% on a public dataset. Additionally, we implemented our approach as a modular ROS package that makes it straightforward for integration in arbitrary graph-based SLAM methods.