A Survey of Body and Face Motion: Datasets, Performance Evaluation Metrics and Generative Techniques

TL;DR

This survey comprehensively reviews the state-of-the-art in body and face motion generation, including datasets, evaluation metrics, and generative techniques, highlighting future challenges for creating realistic and expressive avatars.

Contribution

It is the first extensive review covering both body and face motion generation, integrating core concepts, datasets, and evaluation methods in a single resource.

Findings

Highlights the complexity of generating expressive motion due to verbal and non-verbal cues.

Identifies key datasets and evaluation metrics used in motion generation research.

Proposes future directions for improving realism and coherence in avatar motion synthesis.

Abstract

Body and face motion play an integral role in communication. They convey crucial information on the participants. Advances in generative modeling and multi-modal learning have enabled motion generation from signals such as speech, conversational context and visual cues. However, generating expressive and coherent face and body dynamics remains challenging due to the complex interplay of verbal / non-verbal cues and individual personality traits. This survey reviews body and face motion generation, covering core concepts, representations techniques, generative approaches, datasets and evaluation metrics. We highlight future directions to enhance the realism, coherence and expressiveness of avatars in dyadic settings. To the best of our knowledge, this work is the first comprehensive review to cover both body and face motion. Detailed resources are listed on https://lownish23csz0010.github.io/mogen/.