Mind over Space: Can Multimodal Large Language Models Mentally Navigate?

TL;DR

This paper introduces Video2Mental, a benchmark for evaluating mental navigation in multimodal large language models, and proposes NavMind, a reasoning model that significantly improves spatial planning and navigation over existing models.

Contribution

The paper presents a new benchmark for mental navigation in MLLMs and introduces NavMind, a novel reasoning model with explicit cognitive maps that enhances spatial reasoning and planning.

Findings

Standard pre-training does not induce mental navigation capabilities.

NavMind outperforms existing models in structured spatial representation.

NavMind maintains higher planning accuracy over extended horizons.

Abstract

Despite the widespread adoption of MLLMs in embodied agents, their capabilities remain largely confined to reactive planning from immediate observations, consistently failing in spatial reasoning across extensive spatiotemporal scales. Cognitive science reveals that Biological Intelligence (BI) thrives on "mental navigation": the strategic construction of spatial representations from experience and the subsequent mental simulation of paths prior to action. To bridge the gap between AI and BI, we introduce Video2Mental, a pioneering benchmark for evaluating the mental navigation capabilities of MLLMs. The task requires constructing hierarchical cognitive maps from long egocentric videos and generating landmark-based path plans step by step, with planning accuracy verified through simulator-based physical interaction. Our benchmarking results reveal that mental navigation capability does not naturally emerge from standard pre-training. Frontier MLLMs struggle profoundly with zero-shot structured spatial representation, and their planning accuracy decays precipitously over extended horizons. To overcome this, we propose \textbf{NavMind}, a reasoning model that internalizes mental navigation using explicit, fine-grained cognitive maps as learnable intermediate representations. Through a difficulty-stratified progressive supervised fine-tuning paradigm, NavMind effectively bridges the gap between raw perception and structured planning. Experiments demonstrate that NavMind achieves superior mental navigation capabilities, significantly outperforming frontier commercial and spatial MLLMs.