SkeNa: Learning to Navigate Unseen Environments Based on Abstract Hand-Drawn Maps

TL;DR

This paper introduces SkeNa, a navigation task where agents use hand-drawn sketch maps to reach goals in unseen environments, supported by a large dataset and a novel navigation framework that significantly outperforms previous methods.

Contribution

The paper presents SkeNa, a new navigation task based on sketch maps, along with the SoR dataset and SkeNavigator framework, advancing research in map-based embodied navigation.

Findings

SkeNavigator improves SPL by 105% on high-abstract validation set.

The large-scale SoR dataset contains 54k trajectory and sketch map pairs.

SkeNa outperforms prior floor plan navigation methods significantly.

Abstract

A typical human strategy for giving navigation guidance is to sketch route maps based on the environmental layout. Inspired by this, we introduce Sketch map-based visual Navigation (SkeNa), an embodied navigation task in which an agent must reach a goal in an unseen environment using only a hand-drawn sketch map as guidance. To support research for SkeNa, we present a large-scale dataset named SoR, comprising 54k trajectory and sketch map pairs across 71 indoor scenes. In SoR, we introduce two navigation validation sets with varying levels of abstraction in hand-drawn sketches, categorized based on their preservation of spatial scales in the environment, to facilitate future research. To construct SoR, we develop an automated sketch-generation pipeline that efficiently converts floor plans into hand-drawn representations. To solve SkeNa, we propose SkeNavigator, a navigation framework that aligns visual observations with hand-drawn maps to estimate navigation targets. It employs a Ray-based Map Descriptor (RMD) to enhance sketch map valid feature representation using equidistant sampling points and boundary distances. To improve alignment with visual observations, a Dual-Map Aligned Goal Predictor (DAGP) leverages the correspondence between sketch map features and on-site constructed exploration map features to predict goal position and guide navigation. SkeNavigator outperforms prior floor plan navigation methods by a large margin, improving SPL on the high-abstract validation set by 105% relatively. Our code and dataset will be released.