Benchmarking ground truth trajectories with robotic total stations

TL;DR

This paper compares ground truth trajectory generation methods for SLAM benchmarking, demonstrating that robotic total stations (RTS) provide more precise and reproducible results than GNSS systems in outdoor environments.

Contribution

The study provides empirical evidence that RTS setups outperform GNSS in accuracy and repeatability for outdoor ground truth generation in SLAM benchmarking.

Findings

RTS median disparity: 8.6 mm

GNSS median disparity: 10.6 cm

RTS offers higher precision and reproducibility

Abstract

Benchmarks stand as vital cornerstones in elevating SLAM algorithms within mobile robotics. Consequently, ensuring accurate and reproducible ground truth generation is vital for fair evaluation. A majority of outdoor ground truths are generated by GNSS, which can lead to discrepancies over time, especially in covered areas. However, research showed that RTS setups are more precise and can alternatively be used to generate these ground truths. In our work, we compare both RTS and GNSS systems' precision and repeatability through a set of experiments conducted weeks and months apart in the same area. We demonstrated that RTS setups give more reproducible results, with disparities having a median value of 8.6 mm compared to a median value of 10.6 cm coming from a GNSS setup. These results highlight that RTS can be considered to benchmark process for SLAM algorithms with higher precision.