An Efficient System for Automatic Map Storytelling -- A Case Study on Historical Maps
Ziyi Liu, Claudio Affolter, Sidi Wu, Yizi Chen, Lorenz Hurni

TL;DR
This paper introduces a lightweight, adaptable system that combines fine-tuned CLIP and GPT-3.5 to automatically generate descriptive and storytelling captions for historical maps, overcoming challenges of map complexity and limited training data.
Contribution
It presents a novel decision tree architecture and a hybrid captioning approach that enhances map understanding and storytelling, with minimal fine-tuning and high adaptability.
Findings
System effectively generates relevant map captions.
Invariance to text alterations in maps demonstrated.
Open-source implementation available for extension.
Abstract
Historical maps provide valuable information and knowledge about the past. However, as they often feature non-standard projections, hand-drawn styles, and artistic elements, it is challenging for non-experts to identify and interpret them. While existing image captioning methods have achieved remarkable success on natural images, their performance on maps is suboptimal as maps are underrepresented in their pre-training process. Despite the recent advance of GPT-4 in text recognition and map captioning, it still has a limited understanding of maps, as its performance wanes when texts (e.g., titles and legends) in maps are missing or inaccurate. Besides, it is inefficient or even impractical to fine-tune the model with users' own datasets. To address these problems, we propose a novel and lightweight map-captioning counterpart. Specifically, we fine-tune the state-of-the-art…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsGeographic Information Systems Studies
MethodsRefunds@Expedia|||How do I get a full refund from Expedia? · Attention Is All You Need · Linear Layer · Position-Wise Feed-Forward Layer · Cosine Annealing · Attention Dropout · Softmax · Multi-Head Attention · {Dispute@FaQ-s}How to file a dispute with Expedia? · Linear Warmup With Cosine Annealing
