Studying Illustrations in Manuscripts: An Efficient Deep-Learning Approach

TL;DR

This paper introduces a scalable AI pipeline that leverages deep learning to detect, extract, and analyze illustrations in illuminated manuscripts, enabling large-scale visual analysis for humanities research.

Contribution

It presents a novel, integrated deep-learning framework for large-scale visual analysis of manuscripts, combining detection, extraction, and multimodal description for scholarly exploration.

Findings

Effective detection and extraction of manuscript illustrations.

Identification of visual patterns and cross-manuscript relationships.

Facilitation of large-scale visual scholarship in humanities.

Abstract

The recent Artificial Intelligence (AI) revolution has opened transformative possibilities for the humanities, particularly in unlocking the visual-artistic content embedded in historical illuminated manuscripts. While digital archives now offer unprecedented access to these materials, the ability to systematically locate, extract, and analyze illustrations at scale remains a major challenge. We present a general and scalable AI-based pipeline for large-scale visual analysis of illuminated manuscripts. The framework integrates modern deep-learning models for page-level illustration detection, illustration extraction, and multimodal description, enabling scholars to search, cluster, and study visual materials and artistic trends across entire corpora. We demonstrate the applicability of this approach on large heterogeneous collections, including the Vatican Library and richly illuminated manuscripts such as the Bible of Borso d'Este. The system reveals meaningful visual patterns and cross-manuscript relationships by embedding illustrations into a shared representation space and analyzing their similarity structure (see figure 4). By harnessing recent advances in computer vision and vision-language models, our framework enables new forms of large-scale visual scholarship in historical studies, art history, and cultural heritage making it possible to explore iconography, stylistic trends, and cultural connections in ways that were previously impractical.