Establishing a Unified Evaluation Framework for Human Motion Generation: A Comparative Analysis of Metrics

TL;DR

This paper reviews existing metrics for human motion generation, proposes a unified evaluation framework with a new diversity metric, and demonstrates its application through experiments on three models.

Contribution

It introduces a standardized evaluation setup and a novel diversity metric, improving consistency and interpretability in human motion generation assessment.

Findings

Unified evaluation framework facilitates consistent comparisons.

New diversity metric captures temporal warping variations.

Experimental analysis provides insights into metric interpretation.

Abstract

The development of generative artificial intelligence for human motion generation has expanded rapidly, necessitating a unified evaluation framework. This paper presents a detailed review of eight evaluation metrics for human motion generation, highlighting their unique features and shortcomings. We propose standardized practices through a unified evaluation setup to facilitate consistent model comparisons. Additionally, we introduce a novel metric that assesses diversity in temporal distortion by analyzing warping diversity, thereby enhancing the evaluation of temporal data. We also conduct experimental analyses of three generative models using a publicly available dataset, offering insights into the interpretation of each metric in specific case scenarios. Our goal is to offer a clear, user-friendly evaluation framework for newcomers, complemented by publicly accessible code.