Self-supervised Video Instance Segmentation Can Boost Geographic Entity Alignment in Historical Maps

TL;DR

This paper introduces a self-supervised video instance segmentation approach to improve geographic entity alignment in historical maps, reducing manual annotation needs and enhancing accuracy in linking map features across datasets.

Contribution

The study presents a novel self-supervised training method for video instance segmentation applied to historical maps, including synthetic video generation for pretraining.

Findings

24.9% improvement in AP over baseline

0.23 increase in F1 score

Effective reduction in manual annotation requirements

Abstract

Tracking geographic entities from historical maps, such as buildings, offers valuable insights into cultural heritage, urbanization patterns, environmental changes, and various historical research endeavors. However, linking these entities across diverse maps remains a persistent challenge for researchers. Traditionally, this has been addressed through a two-step process: detecting entities within individual maps and then associating them via a heuristic-based post-processing step. In this paper, we propose a novel approach that combines segmentation and association of geographic entities in historical maps using video instance segmentation (VIS). This method significantly streamlines geographic entity alignment and enhances automation. However, acquiring high-quality, video-format training data for VIS models is prohibitively expensive, especially for historical maps that often contain hundreds or thousands of geographic entities. To mitigate this challenge, we explore self-supervised learning (SSL) techniques to enhance VIS performance on historical maps. We evaluate the performance of VIS models under different pretraining configurations and introduce a novel method for generating synthetic videos from unlabeled historical map images for pretraining. Our proposed self-supervised VIS method substantially reduces the need for manual annotation. Experimental results demonstrate the superiority of the proposed self-supervised VIS approach, achieving a 24.9\% improvement in AP and a 0.23 increase in F1 score compared to the model trained from scratch.