MapSAM: Adapting Segment Anything Model for Automated Feature Detection in Historical Maps

TL;DR

MapSAM is a novel adaptation of the Segment Anything Model that enables automated, prompt-free segmentation of historical maps by employing efficient fine-tuning and automatic prompt generation, significantly reducing manual effort.

Contribution

This paper introduces MapSAM, a parameter-efficient fine-tuning approach that transforms SAM into a fully automated, versatile tool for historical map feature detection with minimal data.

Findings

Effective segmentation of linear and areal features in historical maps.

Performs well with extremely limited training data (as few as 10 samples).

Outperforms baseline methods in adapting to diverse map features.

Abstract

Automated feature detection in historical maps can significantly accelerate the reconstruction of the geospatial past. However, this process is often constrained by the time-consuming task of manually digitizing sufficient high-quality training data. The emergence of visual foundation models, such as the Segment Anything Model (SAM), offers a promising solution due to their remarkable generalization capabilities and rapid adaptation to new data distributions. Despite this, directly applying SAM in a zero-shot manner to historical map segmentation poses significant challenges, including poor recognition of certain geospatial features and a reliance on input prompts, which limits its ability to be fully automated. To address these challenges, we introduce MapSAM, a parameter-efficient fine-tuning strategy that adapts SAM into a prompt-free and versatile solution for various downstream historical map segmentation tasks. Specifically, we employ Weight-Decomposed Low-Rank Adaptation (DoRA) to integrate domain-specific knowledge into the image encoder. Additionally, we develop an automatic prompt generation process, eliminating the need for manual input. We further enhance the positional prompt in SAM, transforming it into a higher-level positional-semantic prompt, and modify the cross-attention mechanism in the mask decoder with masked attention for more effective feature aggregation. The proposed MapSAM framework demonstrates promising performance across two distinct historical map segmentation tasks: one focused on linear features and the other on areal features. Experimental results show that it adapts well to various features, even when fine-tuned with extremely limited data (e.g. 10 shots).