Generating Emotive Gaits for Virtual Agents Using Affect-Based Autoregression

TL;DR

This paper introduces a novel autoregression network that synthesizes emotive virtual agent gaits, enabling real-time emotion expression and transition in interactive augmented reality environments, validated by human perception studies.

Contribution

The work presents a new affect-based autoregression model for generating natural and emotionally expressive gaits, including dataset augmentation and real-time AR integration.

Findings

89% user satisfaction with gait naturalness

Generated gaits accurately conveyed intended emotions

Augmented dataset supports future emotion prediction research

Abstract

We present a novel autoregression network to generate virtual agents that convey various emotions through their walking styles or gaits. Given the 3D pose sequences of a gait, our network extracts pertinent movement features and affective features from the gait. We use these features to synthesize subsequent gaits such that the virtual agents can express and transition between emotions represented as combinations of happy, sad, angry, and neutral. We incorporate multiple regularizations in the training of our network to simultaneously enforce plausible movements and noticeable emotions on the virtual agents. We also integrate our approach with an AR environment using a Microsoft HoloLens and can generate emotive gaits at interactive rates to increase the social presence. We evaluate how human observers perceive both the naturalness and the emotions from the generated gaits of the virtual agents in a web-based study. Our results indicate around 89% of the users found the naturalness of the gaits satisfactory on a five-point Likert scale, and the emotions they perceived from the virtual agents are statistically similar to the intended emotions of the virtual agents. We also use our network to augment existing gait datasets with emotive gaits and will release this augmented dataset for future research in emotion prediction and emotive gait synthesis. Our project website is available at https://gamma.umd.edu/gen_emotive_gaits/.