Signage-Aware Exploration in Open World using Venue Maps

TL;DR

This paper presents a signage-aware exploration system for robots in open-world environments, leveraging venue maps and advanced text recognition to improve search efficiency and accuracy in complex indoor spaces.

Contribution

It introduces a novel signage understanding method with diffusion-based text retrieval and a venue map-guided exploration planner, addressing recognition challenges and environmental discrepancies.

Findings

Outperforms state-of-the-art text spotting methods in recognition accuracy.

Enhances exploration efficiency in large-scale shopping malls.

Effectively balances exploration and exploitation using venue map priors.

Abstract

Current exploration methods struggle to search for shops or restaurants in unknown open-world environments due to the lack of prior knowledge. Humans can leverage venue maps that offer valuable scene priors to aid exploration planning by correlating the signage in the scene with landmark names on the map. However, arbitrary shapes and styles of the texts on signage, along with multi-view inconsistencies, pose significant challenges for robots to recognize them accurately. Additionally, discrepancies between real-world environments and venue maps hinder the integration of text-level information into the planners. This paper introduces a novel signage-aware exploration system to address these challenges, enabling the robots to utilize venue maps effectively. We propose a signage understanding method that accurately detects and recognizes the texts on signage using a diffusion-based text instance retrieval method combined with a 2D-to-3D semantic fusion strategy. Furthermore, we design a venue map-guided exploration-exploitation planner that balances exploration in unknown regions using directional heuristics derived from venue maps and exploitation to get close and adjust orientation for better recognition. Experiments in large-scale shopping malls demonstrate our method's superior signage recognition performance and search efficiency, surpassing state-of-the-art text spotting methods and traditional exploration approaches. Project website: https://sites.google.com/view/signage-aware-exploration.