Efficient Emotion-Aware Iconic Gesture Prediction for Robot Co-Speech

TL;DR

This paper presents a lightweight transformer model that predicts emotion-aware iconic gestures from text and emotion, enhancing robot co-speech communication without needing audio input.

Contribution

The novel model integrates semantic emphasis into gesture prediction, outperforming GPT-4o in classification and regression tasks while being suitable for real-time deployment.

Findings

Outperforms GPT-4o in gesture placement classification

Achieves better intensity regression results

Remains computationally compact for real-time use

Abstract

Co-speech gestures increase engagement and improve speech understanding. Most data-driven robot systems generate rhythmic beat-like motion, yet few integrate semantic emphasis. To address this, we propose a lightweight transformer that derives iconic gesture placement and intensity from text and emotion alone, requiring no audio input at inference time. The model outperforms GPT-4o in both semantic gesture placement classification and intensity regression on the BEAT2 dataset, while remaining computationally compact and suitable for real-time deployment on embodied agents.