Artificial Intelligence and Misinformation in Art: Can Vision Language Models Judge the Hand or the Machine Behind the Canvas?

TL;DR

This paper investigates the limitations of vision language models in art attribution and AI-generated image detection, highlighting their current inability to reliably identify artists or AI-created artworks, which impacts misinformation prevention.

Contribution

It provides an experimental analysis of state-of-the-art AI models' capabilities in art attribution and AI image detection, revealing significant limitations in these tasks.

Findings

Vision language models struggle with artist attribution.

AI models have limited ability to detect AI-generated images.

Results emphasize the need for improved models to prevent misinformation.

Abstract

The attribution of artworks in general and of paintings in particular has always been an issue in art. The advent of powerful artificial intelligence models that can generate and analyze images creates new challenges for painting attribution. On the one hand, AI models can create images that mimic the style of a painter, which can be incorrectly attributed, for example, by other AI models. On the other hand, AI models may not be able to correctly identify the artist for real paintings, inducing users to incorrectly attribute paintings. In this paper, both problems are experimentally studied using state-of-the-art AI models for image generation and analysis on a large dataset with close to 40,000 paintings from 128 artists. The results show that vision language models have limited capabilities to: 1) perform canvas attribution and 2) to identify AI generated images. As users increasingly rely on queries to AI models to get information, these results show the need to improve the capabilities of VLMs to reliably perform artist attribution and detection of AI generated images to prevent the spread of incorrect information.