NPRportrait 1.0: A Three-Level Benchmark for Non-Photorealistic Rendering of Portraits

TL;DR

This paper introduces NPRportrait 1.0, a comprehensive three-level benchmark dataset and evaluation methodology for assessing non-photorealistic portrait stylisation, addressing the lack of standardized evaluation in the field.

Contribution

It presents a novel structured benchmark dataset and a new evaluation methodology for portrait stylisation algorithms, validated through user studies.

Findings

Benchmark enables consistent evaluation of stylisation methods.

Neural style transfer methods perform competitively on the benchmark.

Traditional NPR approaches show varied results across benchmark levels.

Abstract

Despite the recent upsurge of activity in image-based non-photorealistic rendering (NPR), and in particular portrait image stylisation, due to the advent of neural style transfer, the state of performance evaluation in this field is limited, especially compared to the norms in the computer vision and machine learning communities. Unfortunately, the task of evaluating image stylisation is thus far not well defined, since it involves subjective, perceptual and aesthetic aspects. To make progress towards a solution, this paper proposes a new structured, three level, benchmark dataset for the evaluation of stylised portrait images. Rigorous criteria were used for its construction, and its consistency was validated by user studies. Moreover, a new methodology has been developed for evaluating portrait stylisation algorithms, which makes use of the different benchmark levels as well as annotations provided by user studies regarding the characteristics of the faces. We perform evaluation for a wide variety of image stylisation methods (both portrait-specific and general purpose, and also both traditional NPR approaches and neural style transfer) using the new benchmark dataset.