Synthesis in Style: Semantic Segmentation of Historical Documents using Synthetic Data

TL;DR

This paper introduces a novel method for generating synthetic labeled datasets of historical documents using StyleGAN, enabling effective training of segmentation models with minimal manual annotation.

Contribution

The authors propose a new approach to create synthetic labeled data for historical documents by leveraging StyleGAN's semantic features, improving segmentation performance with less annotation effort.

Findings

Models trained on our synthetic dataset outperform those trained on existing synthetic data.

Our method reduces the need for manual annotation in historical document analysis.

Synthetic data generated by our approach leads to better segmentation accuracy.

Abstract

One of the most pressing problems in the automated analysis of historical documents is the availability of annotated training data. The problem is that labeling samples is a time-consuming task because it requires human expertise and thus, cannot be automated well. In this work, we propose a novel method to construct synthetic labeled datasets for historical documents where no annotations are available. We train a StyleGAN model to synthesize document images that capture the core features of the original documents. While originally, the StyleGAN architecture was not intended to produce labels, it indirectly learns the underlying semantics to generate realistic images. Using our approach, we can extract the semantic information from the intermediate feature maps and use it to generate ground truth labels. To investigate if our synthetic dataset can be used to segment the text in historical documents, we use it to train multiple supervised segmentation models and evaluate their performance. We also train these models on another dataset created by a state-of-the-art synthesis approach to show that the models trained on our dataset achieve better results while requiring even less human annotation effort.