Incremental Semantic Localization using Hierarchical Clustering of Object Association Sets

TL;DR

This paper introduces a hierarchical clustering-based method for semantic localization that leverages 3D object detections and spatial configuration invariance to improve relocalization robustness and accuracy in challenging environments.

Contribution

The authors propose a novel incremental semantic localization approach using hierarchical clustering of object association sets based on spatial compatibility and invariance to global transformations.

Findings

Outperforms prior methods in robustness and accuracy

Effective in dynamic scenes and large viewpoint changes

Handles scenes with repeated object instances

Abstract

We present a novel approach for relocalization or place recognition, a fundamental problem to be solved in many robotics, automation, and AR applications. Rather than relying on often unstable appearance information, we consider a situation in which the reference map is given in the form of localized objects. Our localization framework relies on 3D semantic object detections, which are then associated to objects in the map. Possible pair-wise association sets are grown based on hierarchical clustering using a merge metric that evaluates spatial compatibility. The latter notably uses information about relative object configurations, which is invariant with respect to global transformations. Association sets are furthermore updated and expanded as the camera incrementally explores the environment and detects further objects. We test our algorithm in several challenging situations including dynamic scenes, large view-point changes, and scenes with repeated instances. Our experiments demonstrate that our approach outperforms prior art in terms of both robustness and accuracy.