VISTA: Monocular Segmentation-Based Mapping for Appearance and View-Invariant Global Localization

TL;DR

VISTA is a monocular, segmentation-based localization framework that robustly aligns vehicle positions across different environments and seasons without training, achieving high recall and low memory usage.

Contribution

It introduces a novel, domain-agnostic approach combining segmentation, tracking, and geometric matching for appearance and view-invariant localization.

Findings

Up to 69% improvement in recall over baseline methods.

Maintains a compact map only 0.6% the size of baseline maps.

Capable of real-time operation on resource-constrained platforms.

Abstract

Global localization is critical for autonomous navigation, particularly in scenarios where an agent must localize within a map generated in a different session or by another agent, as agents often have no prior knowledge about the correlation between reference frames. However, this task remains challenging in unstructured environments due to appearance changes induced by viewpoint variation, seasonal changes, spatial aliasing, and occlusions -- known failure modes for traditional place recognition methods. To address these challenges, we propose VISTA (View-Invariant Segmentation-Based Tracking for Frame Alignment), a novel open-set, monocular global localization framework that combines: 1) a front-end, object-based, segmentation and tracking pipeline, followed by 2) a submap correspondence search, which exploits geometric consistencies between environment maps to align vehicle reference frames. VISTA enables consistent localization across diverse camera viewpoints and seasonal changes, without requiring any domain-specific training or finetuning. We evaluate VISTA on seasonal and oblique-angle aerial datasets, achieving up to a 69% improvement in recall over baseline methods. Furthermore, we maintain a compact object-based map that is only 0.6% the size of the most memory-conservative baseline, making our approach capable of real-time implementation on resource-constrained platforms.