TL;DR
SGToolkit is an interactive gesture authoring toolkit for embodied conversational agents that combines neural generation with user controls, producing high-quality, speech-synchronized gestures more efficiently than manual methods.
Contribution
Introduces SGToolkit, a neural-based gesture generation toolkit that allows fine and coarse control, improving quality and efficiency over existing automatic and manual approaches.
Findings
User study shows preference over manual authoring
Generated gestures are human-like and speech-appropriate
Toolkit outperforms automatic methods in quality
Abstract
Non-verbal behavior is essential for embodied agents like social robots, virtual avatars, and digital humans. Existing behavior authoring approaches including keyframe animation and motion capture are too expensive to use when there are numerous utterances requiring gestures. Automatic generation methods show promising results, but their output quality is not satisfactory yet, and it is hard to modify outputs as a gesture designer wants. We introduce a new gesture generation toolkit, named SGToolkit, which gives a higher quality output than automatic methods and is more efficient than manual authoring. For the toolkit, we propose a neural generative model that synthesizes gestures from speech and accommodates fine-level pose controls and coarse-level style controls from users. The user study with 24 participants showed that the toolkit is favorable over manual authoring, and the…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
