SpatiaLoc: Leveraging Multi-Level Spatial Enhanced Descriptors for Cross-Modal Localization

TL;DR

SpatiaLoc introduces a multi-level spatial descriptor framework for cross-modal localization using text and point clouds, significantly improving accuracy by modeling spatial relationships at instance and global levels.

Contribution

The paper proposes a novel coarse-to-fine framework with specialized encoders and localizer, enhancing spatial relationship modeling for improved cross-modal localization.

Findings

Outperforms existing SOTA methods on KITTI360Pose

Effective modeling of spatial relationships improves localization accuracy

Utilizes Bezier curves and frequency domain representations for spatial encoding

Abstract

Cross-modal localization using text and point clouds enables robots to localize themselves via natural language descriptions, with applications in autonomous navigation and interaction between humans and robots. In this task, objects often recur across text and point clouds, making spatial relationships the most discriminative cues for localization. Given this characteristic, we present SpatiaLoc, a framework utilizing a coarse-to-fine strategy that emphasizes spatial relationships at both the instance and global levels. In the coarse stage, we introduce a Bezier Enhanced Object Spatial Encoder (BEOSE) that models spatial relationships at the instance level using quadratic Bezier curves. Additionally, a Frequency Aware Encoder (FAE) generates spatial representations in the frequency domain at the global level. In the fine stage, an Uncertainty Aware Gaussian Fine Localizer (UGFL) regresses 2D positions by modeling predictions as Gaussian distributions with a loss function aware of uncertainty. Extensive experiments on KITTI360Pose demonstrate that SpatiaLoc significantly outperforms existing state-of-the-art (SOTA) methods.