Computational identification of significant actors in paintings through symbols and attributes

TL;DR

This paper presents a deep learning-based system for automatically identifying saints in Christian paintings by analyzing symbols and attributes, aiding in semantic interpretation of artworks.

Contribution

It introduces a novel approach combining CNNs and knowledge databases to identify saints based on symbols, advancing automatic art interpretation.

Findings

Successfully identified saints across six centuries of Christian art.

Demonstrated effectiveness of symbol-based identification in complex artworks.

Lays groundwork for broader semantic analysis of fine art.

Abstract

The automatic analysis of fine art paintings presents a number of novel technical challenges to artificial intelligence, computer vision, machine learning, and knowledge representation quite distinct from those arising in the analysis of traditional photographs. The most important difference is that many realist paintings depict stories or episodes in order to convey a lesson, moral, or meaning. One early step in automatic interpretation and extraction of meaning in artworks is the identifications of figures (actors). In Christian art, specifically, one must identify the actors in order to identify the Biblical episode or story depicted, an important step in understanding the artwork. We designed an automatic system based on deep convolutional neural networks and simple knowledge database to identify saints throughout six centuries of Christian art based in large part upon saints symbols or attributes. Our work represents initial steps in the broad task of automatic semantic interpretation of messages and meaning in fine art.