One-Shot Pose-Driving Face Animation Platform

TL;DR

This paper introduces a refined face animation platform that generates expressive talking head videos from a single reference face, using an improved model and a user-friendly demo interface.

Contribution

It enhances existing face animation models with a Face Locator and Motion Frame, enabling more expressive and consistent one-shot talking head video generation.

Findings

Significantly improved video quality and expressiveness.

Effective generalization to new identities without fine-tuning.

User-friendly demo platform for quick video creation.

Abstract

The objective of face animation is to generate dynamic and expressive talking head videos from a single reference face, utilizing driving conditions derived from either video or audio inputs. Current approaches often require fine-tuning for specific identities and frequently fail to produce expressive videos due to the limited effectiveness of Wav2Pose modules. To facilitate the generation of one-shot and more consecutive talking head videos, we refine an existing Image2Video model by integrating a Face Locator and Motion Frame mechanism. We subsequently optimize the model using extensive human face video datasets, significantly enhancing its ability to produce high-quality and expressive talking head videos. Additionally, we develop a demo platform using the Gradio framework, which streamlines the process, enabling users to quickly create customized talking head videos.