Art Authentication with Vision Transformers

TL;DR

This paper evaluates the effectiveness of Vision Transformers, specifically Swin Transformers, in art authentication tasks, comparing their performance to EfficientNet on datasets of Van Gogh paintings and imitations.

Contribution

It demonstrates that Vision Transformers can outperform traditional CNNs like EfficientNet in detecting art forgeries, especially with imitation datasets.

Findings

Swin Transformers achieved over 85% accuracy on imitation datasets.

EfficientNet performed better overall on the contrast set with diverse artworks.

Vision Transformers show promise in improving computer-based art authentication.

Abstract

In recent years, Transformers, initially developed for language, have been successfully applied to visual tasks. Vision Transformers have been shown to push the state-of-the-art in a wide range of tasks, including image classification, object detection, and semantic segmentation. While ample research has shown promising results in art attribution and art authentication tasks using Convolutional Neural Networks, this paper examines if the superiority of Vision Transformers extends to art authentication, improving, thus, the reliability of computer-based authentication of artworks. Using a carefully compiled dataset of authentic paintings by Vincent van Gogh and two contrast datasets, we compare the art authentication performances of Swin Transformers with those of EfficientNet. Using a standard contrast set containing imitations and proxies (works by painters with styles closely related to van Gogh), we find that EfficientNet achieves the best performance overall. With a contrast set that only consists of imitations, we find the Swin Transformer to be superior to EfficientNet by achieving an authentication accuracy of over 85%. These results lead us to conclude that Vision Transformers represent a strong and promising contender in art authentication, particularly in enhancing the computer-based ability to detect artistic imitations.