Quality assessment of 3D human animation: Subjective and objective evaluation

TL;DR

This paper introduces a novel data-driven quality assessment measure for 3D virtual human animations, combining subjective user evaluations with machine learning to improve accuracy over existing methods.

Contribution

It presents the first quality assessment framework for non-parametric 3D virtual human animations using a new dataset and a simple linear regression model.

Findings

Linear regressor achieves 90% correlation with subjective scores.

Outperforms existing deep learning baseline.

Provides a new benchmark for animation quality evaluation.

Abstract

Virtual human animations have a wide range of applications in virtual and augmented reality. While automatic generation methods of animated virtual humans have been developed, assessing their quality remains challenging. Recently, approaches introducing task-oriented evaluation metrics have been proposed, leveraging neural network training. However, quality assessment measures for animated virtual humans that are not generated with parametric body models have yet to be developed. In this context, we introduce a first such quality assessment measure leveraging a novel data-driven framework. First, we generate a dataset of virtual human animations together with their corresponding subjective realism evaluation scores collected with a user study. Second, we use the resulting dataset to learn predicting perceptual evaluation scores. Results indicate that training a linear regressor on our dataset results in a correlation of 90%, which outperforms a state of the art deep learning baseline.