FACTS: Facial Animation Creation using the Transfer of Styles

TL;DR

This paper introduces FACTS, a method that uses StarGAN to transfer styles and emotions onto existing 3D facial animations while preserving lip-sync, reducing the need for costly manual animation or capture.

Contribution

It presents a novel style transfer approach for facial animations using StarGAN with a viseme-preserving loss, enabling emotion and style modifications without losing lip-sync accuracy.

Findings

Successfully transfers emotion and style to facial animations

Maintains lip-sync accuracy during style transfer

Reduces manual effort in facial animation creation

Abstract

The ability to accurately capture and express emotions is a critical aspect of creating believable characters in video games and other forms of entertainment. Traditionally, this animation has been achieved with artistic effort or performance capture, both requiring costs in time and labor. More recently, audio-driven models have seen success, however, these often lack expressiveness in areas not correlated to the audio signal. In this paper, we present a novel approach to facial animation by taking existing animations and allowing for the modification of style characteristics. Specifically, we explore the use of a StarGAN to enable the conversion of 3D facial animations into different emotions and person-specific styles. We are able to maintain the lip-sync of the animations with this method thanks to the use of a novel viseme-preserving loss.