Mr. Virgil: Learning Multi-robot Visual-range Relative Localization

TL;DR

Mr. Virgil introduces an end-to-end learning framework combining graph neural networks and pose graph optimization for robust multi-robot relative localization using UWB and vision, outperforming traditional methods in diverse scenarios.

Contribution

This work presents a novel integrated system that improves multi-robot localization accuracy and robustness through a graph neural network and differentiable pose graph optimization.

Findings

Enhanced localization accuracy in simulation and real-world tests.

Robust data association under occlusion and non-occlusion conditions.

Decentralized implementation enables real-world deployment.

Abstract

Ultra-wideband (UWB)-vision fusion localization has achieved extensive applications in the domain of multi-agent relative localization. The challenging matching problem between robots and visual detection renders existing methods highly dependent on identity-encoded hardware or delicate tuning algorithms. Overconfident yet erroneous matches may bring about irreversible damage to the localization system. To address this issue, we introduce Mr. Virgil, an end-to-end learning multi-robot visual-range relative localization framework, consisting of a graph neural network for data association between UWB rangings and visual detections, and a differentiable pose graph optimization (PGO) back-end. The graph-based front-end supplies robust matching results, accurate initial position predictions, and credible uncertainty estimates, which are subsequently integrated into the PGO back-end to elevate the accuracy of the final pose estimation. Additionally, a decentralized system is implemented for real-world applications. Experiments spanning varying robot numbers, simulation and real-world, occlusion and non-occlusion conditions showcase the stability and exactitude under various scenes compared to conventional methods. Our code is available at: https://github.com/HiOnes/Mr-Virgil.