BEVMAPMATCH: Multimodal BEV Neural Map Matching for Robust Re-Localization of Autonomous Vehicles

TL;DR

BEVMapMatch is a novel multimodal BEV-based re-localization framework for autonomous vehicles operating in GNSS-challenged environments, leveraging lidar-camera fusion and cross-attention for robust global localization.

Contribution

It introduces a context-aware lidar+camera fusion method and a cross-attention search mechanism for accurate GNSS-free vehicle re-localization using BEV segmentation maps.

Findings

Outperforms existing re-localization methods in GNSS-denied environments.

Achieves a Recall@1m of 39.8%, nearly twice the baseline performance.

Utilizes multi-frame BEV segmentation to enhance localization accuracy.

Abstract

Localization in GNSS-denied and GNSS-degraded environments is a challenge for the safe widespread deployment of autonomous vehicles. Such GNSS-challenged environments require alternative methods for robust localization. In this work, we propose BEVMapMatch, a framework for robust vehicle re-localization on a known map without the need for GNSS priors. BEVMapMatch uses a context-aware lidar+camera fusion method to generate multimodal Bird's Eye View (BEV) segmentations around the ego vehicle in both good and adverse weather conditions. Leveraging a search mechanism based on cross-attention, the generated BEV segmentation maps are then used for the retrieval of candidate map patches for map-matching purposes. Finally, BEVMapMatch uses the top retrieved candidate for finer alignment against the generated BEV segmentation, achieving accurate global localization without the need for GNSS. Multiple frames of generated BEV segmentation further improve localization accuracy. Extensive evaluations show that BEVMapMatch outperforms existing methods for re-localization in GNSS-denied and adverse environments, with a Recall@1m of 39.8%, being nearly twice as much as the best performing re-localization baseline. Our code and data will be made available at https://github.com/ssuralcmu/BEVMapMatch.git.