TL;DR
Text2Gestures is a transformer-based model that generates emotionally expressive, full-body gestures for virtual agents aligned with natural language, considering biomechanical features and agent attributes, achieving state-of-the-art results.
Contribution
The paper introduces a novel transformer-based approach that generates contextually and emotionally appropriate gestures for virtual agents, incorporating biomechanical and agent-specific features.
Findings
Achieves state-of-the-art gesture-text alignment performance.
Participants found generated gestures highly plausible.
Generated gestures strongly correlate with intended emotions.
Abstract
We present Text2Gestures, a transformer-based learning method to interactively generate emotive full-body gestures for virtual agents aligned with natural language text inputs. Our method generates emotionally expressive gestures by utilizing the relevant biomechanical features for body expressions, also known as affective features. We also consider the intended task corresponding to the text and the target virtual agents' intended gender and handedness in our generation pipeline. We train and evaluate our network on the MPI Emotional Body Expressions Database and observe that our network produces state-of-the-art performance in generating gestures for virtual agents aligned with the text for narration or conversation. Our network can generate these gestures at interactive rates on a commodity GPU. We conduct a web-based user study and observe that around 91% of participants indicated…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
