MAOMaps: A Photo-Realistic Benchmark For vSLAM and Map Merging Quality Assessment

TL;DR

MAOMaps provides a photo-realistic benchmark dataset and evaluation tools for assessing the accuracy of vSLAM and map merging algorithms, enabling comprehensive performance analysis in realistic environments.

Contribution

It introduces a novel benchmark with a dataset and tools for automatic evaluation of vSLAM and map merging, including a new method for map correspondence considering SLAM context.

Findings

Provides a photo-realistic dataset with ground truth for localization and mapping

Includes a novel map correspondence method that accounts for SLAM context

Open-sourced and ROS-compatible for community use

Abstract

Running numerous experiments in simulation is a necessary step before deploying a control system on a real robot. In this paper we introduce a novel benchmark that is aimed at quantitatively evaluating the quality of vision-based simultaneous localization and mapping (vSLAM) and map merging algorithms. The benchmark consists of both a dataset and a set of tools for automatic evaluation. The dataset is photo-realistic and provides both the localization and the map ground truth data. This makes it possible to evaluate not only the localization part of the SLAM pipeline but the mapping part as well. To compare the vSLAM-built maps and the ground-truth ones we introduce a novel way to find correspondences between them that takes the SLAM context into account (as opposed to other approaches like nearest neighbors). The benchmark is ROS-compatable and is open-sourced to the community. The data and the code are available at: \texttt{github.com/CnnDepth/MAOMaps}.