Motion Transfer-Enhanced StyleGAN for Generating Diverse Macaque Facial Expressions

TL;DR

This paper introduces a novel method combining data augmentation, sample selection, and loss refinement in StyleGAN2 to generate diverse and realistic macaque facial expressions, aiding neuroscience research.

Contribution

The study presents a new approach that enhances StyleGAN2 with motion transfer and data strategies to generate expressive macaque faces with limited data.

Findings

Generated diverse macaque facial expressions outperform baseline models.

Model effectively disentangles motion components as style parameters.

Enables style-based editing of facial movements.

Abstract

Generating animal faces using generative AI techniques is challenging because the available training images are limited both in quantity and variation, particularly for facial expressions across individuals. In this study, we focus on macaque monkeys, widely studied in systems neuroscience and evolutionary research, and propose a method to generate their facial expressions using a style-based generative image model (i.e., StyleGAN2). To address data limitations, we implemented: 1) data augmentation by synthesizing new facial expression images using a motion transfer to animate still images with computer graphics, 2) sample selection based on the latent representation of macaque faces from an initially trained StyleGAN2 model to ensure the variation and uniform sampling in training dataset, and 3) loss function refinement to ensure the accurate reproduction of subtle movements, such as eye movements. Our results demonstrate that the proposed method enables the generation of diverse facial expressions for multiple macaque individuals, outperforming models trained solely on original still images. Additionally, we show that our model is effective for style-based image editing, where specific style parameters correspond to distinct facial movements. These findings underscore the model's potential for disentangling motion components as style parameters, providing a valuable tool for research on macaque facial expressions.