Task-driven SLAM Benchmarking For Robot Navigation

TL;DR

This paper introduces TaskSLAM-Bench, a new benchmark for SLAM in robot navigation that emphasizes repeatability and real-world performance, providing insights into visual and LiDAR SLAM methods.

Contribution

It proposes a task-driven benchmarking approach focusing on precision, integrating mapping capabilities, and enabling practical testing in simulated and real environments.

Findings

Passive stereo SLAM matches LiDAR SLAM in precision indoors

TaskSLAM-Bench offers a richer assessment of SLAM performance

Benchmark is easy to implement and publicly available

Abstract

A critical use case of SLAM for mobile assistive robots is to support localization during a navigation-based task. Current SLAM benchmarks overlook the significance of repeatability (precision), despite its importance in real-world deployments. To address this gap, we propose a task-driven approach to SLAM benchmarking, TaskSLAM-Bench. It employs precision as a key metric, accounts for SLAM's mapping capabilities, and has easy-to-meet implementation requirements. Simulated and real-world testing scenarios of SLAM methods provide insights into the navigation performance properties of modern visual and LiDAR SLAM solutions. The outcomes show that passive stereo SLAM operates at a level of precision comparable to LiDAR SLAM in typical indoor environments. TaskSLAM-Bench complements existing benchmarks and offers richer assessment of SLAM performance in navigation-focused scenarios. Publicly available code permits in-situ SLAM testing in custom environments with properly equipped robots.