VLG-Loc: Vision-Language Global Localization from Labeled Footprint Maps

TL;DR

VLG-Loc introduces a vision-language based global localization approach that uses labeled footprint maps and a vision-language model to robustly localize robots in changing environments, outperforming scan-based methods.

Contribution

The paper proposes a novel localization method combining vision-language models with footprint maps, enabling robust localization without geometric details.

Findings

Outperforms existing scan-based localization methods in robustness.

Effective in simulated and real-world retail environments.

Improved accuracy through probabilistic fusion of visual and scan data.

Abstract

This paper presents Vision-Language Global Localization (VLG-Loc), a novel global localization method that uses human-readable labeled footprint maps containing only names and areas of distinctive visual landmarks in an environment. While humans naturally localize themselves using such maps, translating this capability to robotic systems remains highly challenging due to the difficulty of establishing correspondences between observed landmarks and those in the map without geometric and appearance details. To address this challenge, VLG-Loc leverages a vision-language model (VLM) to search the robot's multi-directional image observations for the landmarks noted in the map. The method then identifies robot poses within a Monte Carlo localization framework, where the found landmarks are used to evaluate the likelihood of each pose hypothesis. Experimental validation in simulated and real-world retail environments demonstrates superior robustness compared to existing scan-based methods, particularly under environmental changes. Further improvements are achieved through the probabilistic fusion of visual and scan-based localization.