Style Transfer Dataset: What Makes A Good Stylization?

TL;DR

This paper introduces a new dataset for image style transfer, including human ratings and analysis of factors influencing stylization quality, aiming to improve evaluation and development of style transfer methods.

Contribution

The paper provides a novel, annotated dataset for style transfer with analysis of factors affecting stylization quality and discusses a methodology for dataset creation.

Findings

Factors influencing user ratings identified

Quantitative measures correlate with stylization quality

Dataset enables automated evaluation and configuration

Abstract

We present a new dataset with the goal of advancing image style transfer - the task of rendering one image in the style of another image. The dataset covers various content and style images of different size and contains 10.000 stylizations manually rated by three annotators in 1-10 scale. Based on obtained ratings, we find which factors are mostly responsible for favourable and poor user evaluations and show quantitative measures having statistically significant impact on user grades. A methodology for creating style transfer datasets is discussed. Presented dataset can be used in automating multiple tasks, related to style transfer configuration and evaluation.