When Engineering Outruns Intelligence: Rethinking Instruction-Guided Navigation

TL;DR

This paper critically reevaluates instruction-guided navigation systems, revealing that geometric cues play a larger role than language models, and introduces simple, training-free variants that perform competitively.

Contribution

It demonstrates that geometry-based methods can match or outperform LLM-driven approaches in instruction-guided navigation, challenging assumptions about language model necessity.

Findings

FPE matches or exceeds LLM-based instruction following without API calls.

SHF achieves similar accuracy with a smaller language prior.

Geometry accounts for much of the reported progress in instruction-guided navigation.

Abstract

Recent ObjectNav systems credit large language models (LLMs) for sizable zero-shot gains, yet it remains unclear how much comes from language versus geometry. We revisit this question by re-evaluating an instruction-guided pipeline, InstructNav, under a detector-controlled setting and introducing two training-free variants that only alter the action value map: a geometry-only Frontier Proximity Explorer (FPE) and a lightweight Semantic-Heuristic Frontier (SHF) that polls the LLM with simple frontier votes. Across HM3D and MP3D, FPE matches or exceeds the detector-controlled instruction follower while using no API calls and running faster; SHF attains comparable accuracy with a smaller, localized language prior. These results suggest that carefully engineered frontier geometry accounts for much of the reported progress, and that language is most reliable as a light heuristic rather than an end-to-end planner. Code available at: https://github.com/matinaghaei/instructnav-scrutinized