Into the Unknown: Generating Geospatial Descriptions for New Environments
Tzuf Paz-Argaman, John Palowitch, Sayali Kulkarni, Reut Tsarfaty, and, Jason Baldridge

TL;DR
This paper introduces a large-scale data augmentation method using geospatial data and knowledge-graphs to improve the generation of spatial descriptions for new environments, significantly enhancing geolocation accuracy in unseen settings.
Contribution
The authors propose a novel augmentation approach combining CFG templates and LLMs to generate high-quality spatial instructions, improving geospatial reasoning in unfamiliar environments.
Findings
45.83% improvement in 100-meter accuracy on unseen environments
CFG-based augmentation outperforms LLM-based methods
Enhanced spatial reasoning with structured geospatial data
Abstract
Similar to vision-and-language navigation (VLN) tasks that focus on bridging the gap between vision and language for embodied navigation, the new Rendezvous (RVS) task requires reasoning over allocentric spatial relationships (independent of the observer's viewpoint) using non-sequential navigation instructions and maps. However, performance substantially drops in new environments with no training data. Using opensource descriptions paired with coordinates (e.g., Wikipedia) provides training data but suffers from limited spatially-oriented text resulting in low geolocation resolution. We propose a large-scale augmentation method for generating high-quality synthetic data for new environments using readily available geospatial data. Our method constructs a grounded knowledge-graph, capturing entity relationships. Sampled entities and relations (`shop north of school') generate navigation…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
Taxonomy
TopicsGeographic Information Systems Studies · Semantic Web and Ontologies
MethodsFocus
