LangMap: A Hierarchical Benchmark for Open-Vocabulary Goal Navigation

TL;DR

LangMap introduces a comprehensive benchmark for open-vocabulary goal navigation in 3D indoor environments, enabling evaluation of language understanding and navigation at multiple semantic levels.

Contribution

The paper presents LangMap, a large-scale, high-quality benchmark with diverse tasks and annotations for evaluating language-guided embodied navigation.

Findings

Rich context and memory improve navigation success.

Long-tailed, small, and distant goals remain challenging.

Richer descriptions outperform previous benchmarks.

Abstract

The relationships between objects and language are fundamental to meaningful communication between humans and AI, and to practically useful embodied intelligence. We introduce HieraNav, a multi-granularity, open-vocabulary goal navigation task where agents interpret natural language instructions to reach targets at four semantic levels: scene, room, region, and instance. To this end, we present Language as a Map (LangMap), a large-scale benchmark built on real-world 3D indoor scans with comprehensive human-verified annotations and tasks spanning these levels. LangMap provides region labels, discriminative region descriptions, discriminative instance descriptions covering 414 object categories, and over 18K navigation tasks. Each target features both concise and detailed descriptions, enabling evaluation across different instruction styles. LangMap achieves superior annotation quality, outperforming GOAT-Bench by 23.8% in discriminative accuracy using four times fewer words. Comprehensive evaluations of zero-shot and supervised models on LangMap reveal that richer context and memory improve success, while long-tailed, small, context-dependent, and distant goals, as well as multi-goal completion, remain challenging. HieraNav and LangMap establish a rigorous testbed for advancing language-driven embodied navigation. Project: https://bo-miao.github.io/LangMap