Unfolding the Past: A Comprehensive Deep Learning Approach to Analyzing Incunabula Pages

TL;DR

This paper presents a deep learning-based method for analyzing incunabula pages, including object detection, OCR, and image classification, demonstrating high accuracy and potential for digital humanities research.

Contribution

It introduces a new annotated dataset and combines multiple deep learning models to analyze the structure and content of early printed book pages.

Findings

YOLO11n achieved F1=0.94 on custom data

Tesseract OCR outperformed Kraken OCR on Text regions

ResNet18 achieved 98.7% accuracy in classifying illustration types

Abstract

We developed a proof-of-concept method for the automatic analysis of the structure and content of incunabula pages. A custom dataset comprising 500 annotated pages from five different incunabula was created using resources from the Jagiellonian Digital Library. Each page was manually labeled with five predefined classes: Text, Title, Picture, Table, and Handwriting. Additionally, the publicly available DocLayNet dataset was utilized as supplementary training data. To perform object detection, YOLO11n and YOLO11s models were employed and trained using two strategies: a combined dataset (DocLayNet and the custom dataset) and the custom dataset alone. The highest performance (F1 = 0.94) was achieved by the YOLO11n model trained exclusively on the custom data. Optical character recognition was then conducted on regions classified as Text, using both Tesseract and Kraken OCR, with Tesseract demonstrating superior results. Subsequently, image classification was applied to the Picture class using a ResNet18 model, achieving an accuracy of 98.7% across five subclasses: Decorative_letter, Illustration, Other, Stamp, and Wrong_detection. Furthermore, the CLIP model was utilized to generate semantic descriptions of illustrations. The results confirm the potential of machine learning in the analysis of early printed books, while emphasizing the need for further advancements in OCR performance and visual content interpretation.