Intentional Gesture: Deliver Your Intentions with Gestures for Speech
Pinxin Liu, Haiyang Liu, Luchuan Song, Jason J. Corso, Chenliang Xu

TL;DR
This paper presents Intentional-Gesture, a framework for generating semantically meaningful gestures aligned with speech by reasoning about high-level communicative intentions, advancing gesture synthesis in digital humans.
Contribution
It introduces a novel intention-aware gesture generation framework, including a new dataset with intention annotations and a motion tokenizer that incorporates high-level communicative functions.
Findings
Achieves state-of-the-art performance on BEAT-2 benchmark.
Demonstrates improved semantic relevance of generated gestures.
Provides a modular foundation for expressive gesture synthesis.
Abstract
When humans speak, gestures help convey communicative intentions, such as adding emphasis or describing concepts. However, current co-speech gesture generation methods rely solely on superficial linguistic cues (e.g. speech audio or text transcripts), neglecting to understand and leverage the communicative intention that underpins human gestures. This results in outputs that are rhythmically synchronized with speech but are semantically shallow. To address this gap, we introduce Intentional-Gesture, a novel framework that casts gesture generation as an intention-reasoning task grounded in high-level communicative functions. First, we curate the InG dataset by augmenting BEAT-2 with gesture-intention annotations (i.e., text sentences summarizing intentions), which are automatically annotated using large vision-language models. Next, we introduce the Intentional Gesture Motion Tokenizer…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsHearing Impairment and Communication · Language, Metaphor, and Cognition · Language, Discourse, Communication Strategies
