It's A Match! Gesture Generation Using Expressive Parameter Matching

TL;DR

This paper presents a database-driven gesture generation method that selects gestures based on speech-related expressive parameters, resulting in more appropriate speech-gesture matching than baseline approaches.

Contribution

It introduces a novel gesture selection approach using expressive parameters linked to speech, improving the naturalness of generated gestures.

Findings

Gestures selected by our method were rated significantly better in speech match.

The method outperformed baseline approaches in perceptual evaluations.

Expressive parameters like velocity and finger extension are key for matching gestures to speech.

Abstract

Automatic gesture generation from speech generally relies on implicit modelling of the nondeterministic speech-gesture relationship and can result in averaged motion lacking defined form. Here, we propose a database-driven approach of selecting gestures based on specific motion characteristics that have been shown to be associated with the speech audio. We extend previous work that identified expressive parameters of gesture motion that can both be predicted from speech and are perceptually important for a good speech-gesture match, such as gesture velocity and finger extension. A perceptual study was performed to evaluate the appropriateness of the gestures selected with our method. We compare our method with two baseline selection methods. The first respects timing, the desired onset and duration of a gesture, but does not match gesture form in other ways. The second baseline additionally disregards the original gesture timing for selecting gestures. The gesture sequences from our method were rated as a significantly better match to the speech than gestures selected by either baseline method.