MapVerse: A Benchmark for Geospatial Question Answering on Diverse Real-World Maps
Sharat Bhat, Harshita Khandelwal, Tushar Kataria, Vivek Gupta

TL;DR
MapVerse is a comprehensive benchmark dataset with real-world maps and diverse questions designed to evaluate and improve models' geospatial reasoning capabilities, highlighting current limitations in spatial understanding.
Contribution
The paper introduces MapVerse, a large-scale, real-world map dataset with diverse questions, providing a new standard for evaluating geospatial reasoning in multimodal models.
Findings
Current models perform well on simple classification tasks.
Models struggle with complex spatial reasoning tasks.
Fine-grained analysis reveals specific reasoning gaps.
Abstract
Maps are powerful carriers of structured and contextual knowledge, encompassing geography, demographics, infrastructure, and environmental patterns. Reasoning over such knowledge requires models to integrate spatial relationships, visual cues, real-world context, and domain-specific expertise-capabilities that current large language models (LLMs) and vision-language models (VLMs) still struggle to exhibit consistently. Yet, datasets used to benchmark VLMs on map-based reasoning remain narrow in scope, restricted to specific domains, and heavily reliant on artificially generated content (outputs from LLMs or pipeline-based methods), offering limited depth for evaluating genuine geospatial reasoning. To address this gap, we present MapVerse, a large-scale benchmark built on real-world maps. It comprises 11,837 human-authored question-answer pairs across 1,025 maps, spanning ten diverse…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsGeographic Information Systems Studies · Multimodal Machine Learning Applications · Constraint Satisfaction and Optimization
