Panoptic-SLAM: Visual SLAM in Dynamic Environments using Panoptic Segmentation

TL;DR

Panoptic-SLAM introduces a robust visual SLAM system that leverages panoptic segmentation to effectively handle dynamic environments and unknown moving objects, outperforming existing methods in accuracy and real-world applicability.

Contribution

This work presents the first SLAM system using panoptic segmentation to filter dynamic objects, enhancing robustness in unknown and dynamic scenes.

Findings

Four times more accurate than PVO on average

Effective in real-world scenarios with unknown objects

Validated with real robot experiments and motion capture

Abstract

The majority of visual SLAM systems are not robust in dynamic scenarios. The ones that deal with dynamic objects in the scenes usually rely on deep-learning-based methods to detect and filter these objects. However, these methods cannot deal with unknown moving objects. This work presents Panoptic-SLAM, an open-source visual SLAM system robust to dynamic environments, even in the presence of unknown objects. It uses panoptic segmentation to filter dynamic objects from the scene during the state estimation process. Panoptic-SLAM is based on ORB-SLAM3, a state-of-the-art SLAM system for static environments. The implementation was tested using real-world datasets and compared with several state-of-the-art systems from the literature, including DynaSLAM, DS-SLAM, SaD-SLAM, PVO and FusingPanoptic. For example, Panoptic-SLAM is on average four times more accurate than PVO, the most recent panoptic-based approach for visual SLAM. Also, experiments were performed using a quadruped robot with an RGB-D camera to test the applicability of our method in real-world scenarios. The tests were validated by a ground-truth created with a motion capture system.