A Multi-modal Registration and Visualization Software Tool for Artworks using CraquelureNet

TL;DR

This paper introduces a software tool that uses a convolutional neural network to automatically register and visualize multi-modal images of artworks, aiding art analysis with efficient and adaptable comparison of different imaging techniques.

Contribution

The paper presents a novel multi-modal registration tool with a CNN-based feature extractor and an interactive GUI for art investigations, demonstrating effectiveness on paintings and prints.

Findings

High registration accuracy on multi-modal paintings

Fast inference time suitable for practical use

Effective transferability to historical prints

Abstract

For art investigations of paintings, multiple imaging technologies, such as visual light photography, infrared reflectography, ultraviolet fluorescence photography, and x-radiography are often used. For a pixel-wise comparison, the multi-modal images have to be registered. We present a registration and visualization software tool, that embeds a convolutional neural network to extract cross-modal features of the crack structures in historical paintings for automatic registration. The graphical user interface processes the user's input to configure the registration parameters and to interactively adapt the image views with the registered pair and image overlays, such as by individual or synchronized zoom or movements of the views. In the evaluation, we qualitatively and quantitatively show the effectiveness of our software tool in terms of registration performance and short inference time on multi-modal paintings and its transferability by applying our method to historical prints.