Can LLMs Learn to Map the World from Local Descriptions?

TL;DR

This paper explores the ability of Large Language Models to develop global spatial understanding from local descriptions, enabling tasks like layout inference and navigation in simulated environments.

Contribution

It demonstrates that LLMs can infer global spatial layouts and learn road connectivity from fragmented local data, a novel capability in spatial cognition modeling.

Findings

LLMs generalize to unseen spatial relationships.

Latent representations align with real-world spatial distributions.

LLMs enable accurate path planning and navigation.

Abstract

Recent advances in Large Language Models (LLMs) have demonstrated strong capabilities in tasks such as code and mathematics. However, their potential to internalize structured spatial knowledge remains underexplored. This study investigates whether LLMs, grounded in locally relative human observations, can construct coherent global spatial cognition by integrating fragmented relational descriptions. We focus on two core aspects of spatial cognition: spatial perception, where models infer consistent global layouts from local positional relationships, and spatial navigation, where models learn road connectivity from trajectory data and plan optimal paths between unconnected locations. Experiments conducted in a simulated urban environment demonstrate that LLMs not only generalize to unseen spatial relationships between points of interest (POIs) but also exhibit latent representations aligned with real-world spatial distributions. Furthermore, LLMs can learn road connectivity from trajectory descriptions, enabling accurate path planning and dynamic spatial awareness during navigation.