Learning Portrait Style Representations

TL;DR

This paper explores how neural networks can learn higher-level style features of artwork, using expert annotations, priors, and large-scale portrait datasets to improve style understanding and classification.

Contribution

It introduces a study of style representations incorporating higher-level features, utilizing expert annotations, priors, and a new large-scale portrait dataset for computational analysis.

Findings

Neural networks can learn meaningful higher-level style features.

Expert annotations and priors influence style representation quality.

The approach enables zero-shot artist classification.

Abstract

Style analysis of artwork in computer vision predominantly focuses on achieving results in target image generation through optimizing understanding of low level style characteristics such as brush strokes. However, fundamentally different techniques are required to computationally understand and control qualities of art which incorporate higher level style characteristics. We study style representations learned by neural network architectures incorporating these higher level characteristics. We find variation in learned style features from incorporating triplets annotated by art historians as supervision for style similarity. Networks leveraging statistical priors or pretrained on photo collections such as ImageNet can also derive useful visual representations of artwork. We align the impact of these expert human knowledge, statistical, and photo realism priors on style representations with art historical research and use these representations to perform zero-shot classification of artists. To facilitate this work, we also present the first large-scale dataset of portraits prepared for computational analysis.