SignNav: Leveraging Signage for Semantic Visual Navigation in Large-Scale Indoor Environments

TL;DR

This paper introduces SignNav, a new task for embodied navigation using signage cues in large-scale indoor environments, along with a dataset and a novel transformer-based model that achieves state-of-the-art results.

Contribution

The paper presents the SignNav task, the LSI-Dataset, and the START model, advancing semantic visual navigation by leveraging signage cues and addressing dynamic semantic hints.

Findings

Achieved 80% success rate and 0.74 NDTW on validation set.

Demonstrated real-world deployment without pre-built maps.

Proposed a spatial-temporal transformer for improved decision-making.

Abstract

Humans routinely leverage semantic hints provided by signage to navigate to destinations within novel Large-Scale Indoor (LSI) environments, such as hospitals and airport terminals. However, this capability remains underexplored within the field of embodied navigation. This paper introduces a novel embodied navigation task, SignNav, which requires the agent to interpret semantic hint from signage and reason about the subsequent action based on current observation. To facilitate research in this domain, we construct the LSI-Dataset for the training and evaluation of various SignNav agents. Dynamically changing semantic hints and sparse placement of signage in LSI environments present significant challenges to the SignNav task. To address these challenges, we propose the Spatial-Temporal Aware Transformer (START) model for end-to-end decision-making. The spatial-aware module grounds the semantic hint of signage into physical world, while the temporal-aware module captures long-range dependencies between historical states and current observation. Leveraging a two-stage training strategy with Dataset Aggregation (DAgger), our approach achieves state-of-the-art performance, recording an 80% Success Rate (SR) and 0.74 NDTW on val-unseen split. Real-world deployment further demonstrates the practicality of our method in physical environment without pre-built map.