Evaluation and comparison of eight popular Lidar and Visual SLAM algorithms

TL;DR

This paper evaluates and compares eight popular open-source Lidar and visual SLAM algorithms across various indoor and outdoor conditions, analyzing their accuracy and computational efficiency to guide optimal sensor-algorithm pairing.

Contribution

It provides a comprehensive experimental comparison of eight SLAM algorithms under different environmental and sensor configurations, highlighting their strengths and limitations.

Findings

LIO SAM performs best in outdoor environments.

LOAM shows robustness to terrain and vibration.

Computational resource requirements vary significantly among algorithms.

Abstract

In this paper, we evaluate eight popular and open-source 3D Lidar and visual SLAM (Simultaneous Localization and Mapping) algorithms, namely LOAM, Lego LOAM, LIO SAM, HDL Graph, ORB SLAM3, Basalt VIO, and SVO2. We have devised experiments both indoor and outdoor to investigate the effect of the following items: i) effect of mounting positions of the sensors, ii) effect of terrain type and vibration, iii) effect of motion (variation in linear and angular speed). We compare their performance in terms of relative and absolute pose error. We also provide comparison on their required computational resources. We thoroughly analyse and discuss the results and identify the best performing system for the environment cases with our multi-camera and multi-Lidar indoor and outdoor datasets. We hope our findings help one to choose a sensor and the corresponding SLAM algorithm combination suiting their needs, based on their target environment.