A Cross-Dataset Study for Text-based 3D Human Motion Retrieval

TL;DR

This study investigates cross-dataset generalization in text-based 3D human motion retrieval, revealing dataset biases and the potential of text augmentation, while pioneering zero-shot action recognition on BABEL without categorical labels.

Contribution

It introduces a unified SMPL format for cross-dataset evaluation and presents the first zero-shot action recognition results on BABEL, highlighting domain gaps and future research directions.

Findings

Dataset biases affect cross-dataset generalization.

Text augmentations partially reduce domain gap.

Zero-shot action recognition achieved on BABEL.

Abstract

We provide results of our study on text-based 3D human motion retrieval and particularly focus on cross-dataset generalization. Due to practical reasons such as dataset-specific human body representations, existing works typically benchmarkby training and testing on partitions from the same dataset. Here, we employ a unified SMPL body format for all datasets, which allows us to perform training on one dataset, testing on the other, as well as training on a combination of datasets. Our results suggest that there exist dataset biases in standard text-motion benchmarks such as HumanML3D, KIT Motion-Language, and BABEL. We show that text augmentations help close the domain gap to some extent, but the gap remains. We further provide the first zero-shot action recognition results on BABEL, without using categorical action labels during training, opening up a new avenue for future research.