Streamlined Facial Data Collection based on Utterance and Emotional Data for Human-to-Avatar Reconstruction

TL;DR

This paper presents a new efficient facial data collection method for human-to-avatar reconstruction that uses utterance and emotional data, reducing data requirements while maintaining perceived realism and naturalness.

Contribution

It introduces a two-phase methodology to identify essential facial data for photorealistic avatar reconstruction, emphasizing efficiency and user perception.

Findings

Targeted utterance and emotional data match extensive data in realism and naturalness.

Reduced data collection time and data usage without sacrificing perceived quality.

Practical guidelines for real-time avatar reconstruction in AR/VR applications.

Abstract

This study explores a streamlined facial data collection method for conversational contexts, addressing the limitations of existing approaches that often require extensive datasets and prioritize technical metrics over user perception and experience. We systematically investigate which facial expression data are essential for reconstructing photorealistic avatars and how they can be captured efficiently. Our research employs a two-phase methodology to identify efficient facial data collection strategies and evaluate their effectiveness. In the first phase, we conduct facial data acquisition and evaluate reconstruction performance using utterance data and emotional data. In the second phase, we carry out a comprehensive user evaluation comparing three progressive conditions: utterance only, utterance and emotional data, and a control condition involving extensive data. Findings from 24 participants engaged in simulated face-to-face conversations reveal that targeted utterance and emotional data achieve comparable levels of perceived realism, naturalness, and telepresence, while reducing training time and data usage when compared to the extensive data collection approach. These results demonstrate that targeted data inputs can enable efficient avatar face reconstruction, offering practical guidelines for real-time applications such as AR/VR telepresence and highlighting the trade-off between data quantity and perceived quality.