Multi-Modal Loop Closing in Unstructured Planetary Environments with Visually Enriched Submaps

TL;DR

This paper introduces a multi-modal SLAM approach for planetary rovers that combines visual and depth data from stereo cameras to improve loop closure detection in unstructured environments, validated through real-world tests.

Contribution

It proposes a novel method that fuses visual keyframes and stereo point cloud data for enhanced loop closure detection in planetary exploration.

Findings

Multi-modal approach improves loop closure detection in challenging environments.

Fusion of visual and depth data outperforms single-modal methods.

Validated in both laboratory and planetary analog environments.

Abstract

Future planetary missions will rely on rovers that can autonomously explore and navigate in unstructured environments. An essential element is the ability to recognize places that were already visited or mapped. In this work, we leverage the ability of stereo cameras to provide both visual and depth information, guiding the search and validation of loop closures from a multi-modal perspective. We propose to augment submaps that are created by aggregating stereo point clouds, with visual keyframes. Point clouds matches are found by comparing CSHOT descriptors and validated by clustering, while visual matches are established by comparing keyframes using Bag-of-Words (BoW) and ORB descriptors. The relative transformations resulting from both keyframe and point cloud matches are then fused to provide pose constraints between submaps in our graph-based SLAM framework. Using the LRU rover, we performed several tests in both an indoor laboratory environment as well as a challenging planetary analog environment on Mount Etna, Italy. These environments consist of areas where either keyframes or point clouds alone failed to provide adequate matches demonstrating the benefit of the proposed multi-modal approach.