Can We Teach Computers to Understand Art? Domain Adaptation for Enhancing Deep Networks Capacity to De-Abstract Art

TL;DR

This paper investigates how to improve CNNs' ability to recognize genres in abstract art by applying domain adaptation techniques, revealing that neural style transfer is not the most effective method.

Contribution

It introduces a comprehensive evaluation of domain adaptation methods for art genre recognition and assesses CNNs' capacity to understand abstract artistic representations.

Findings

CNNs achieve high performance in recognizing art genres.

Neural style transfer is less effective than other domain adaptation methods.

CNNs can partially understand abstract art, but face limitations.

Abstract

Humans comprehend a natural scene at a single glance; painters and other visual artists, through their abstract representations, stressed this capacity to the limit. The performance of computer vision solutions matched that of humans in many problems of visual recognition. In this paper we address the problem of recognizing the genre (subject) in digitized paintings using Convolutional Neural Networks (CNN) as part of the more general dealing with abstract and/or artistic representation of scenes. Initially we establish the state of the art performance by training a CNN from scratch. In the next level of evaluation, we identify aspects that hinder the CNNs' recognition, such as artistic abstraction. Further, we test various domain adaptation methods that could enhance the subject recognition capabilities of the CNNs. The evaluation is performed on a database of 80,000 annotated digitized paintings, which is tentatively extended with artistic photographs, either original or stylized, in order to emulate artistic representations. Surprisingly, the most efficient domain adaptation is not the neural style transfer. Finally, the paper provides an experiment-based assessment of the abstraction level that CNNs are able to achieve.