ArtFID: Quantitative Evaluation of Neural Style Transfer

TL;DR

This paper introduces ArtFID, a quantitative metric for evaluating neural style transfer quality, aligning well with human judgments and enabling better comparison of style transfer methods.

Contribution

The paper proposes a new quantitative evaluation metric for neural style transfer that correlates strongly with human perception, addressing a gap in current assessment practices.

Findings

ArtFID correlates highly with human judgment.

The metric enables objective comparison of style transfer models.

Extensive evaluations validate the effectiveness of the proposed method.

Abstract

The field of neural style transfer has experienced a surge of research exploring different avenues ranging from optimization-based approaches and feed-forward models to meta-learning methods. The developed techniques have not just progressed the field of style transfer, but also led to breakthroughs in other areas of computer vision, such as all of visual synthesis. However, whereas quantitative evaluation and benchmarking have become pillars of computer vision research, the reproducible, quantitative assessment of style transfer models is still lacking. Even in comparison to other fields of visual synthesis, where widely used metrics exist, the quantitative evaluation of style transfer is still lagging behind. To support the automatic comparison of different style transfer approaches and to study their respective strengths and weaknesses, the field would greatly benefit from a quantitative measurement of stylization performance. Therefore, we propose a method to complement the currently mostly qualitative evaluation schemes. We provide extensive evaluations and a large-scale user study to show that the proposed metric strongly coincides with human judgment.