Identity-Preserving Pose-Guided Character Animation via Facial Landmarks Transformation

TL;DR

This paper introduces FLT, a method that uses 3D face modeling to improve facial identity preservation in pose-guided character animations, especially in complex scenarios like dancing.

Contribution

The paper presents a novel Facial Landmarks Transformation (FLT) technique that aligns 2D landmarks with a 3D face model to enhance identity preservation in animations.

Findings

FLT improves facial identity consistency in animations.

Experimental results show significant enhancement over existing methods.

The approach effectively handles complex dynamic scenarios.

Abstract

Creating realistic pose-guided image-to-video character animations while preserving facial identity remains challenging, especially in complex and dynamic scenarios such as dancing, where precise identity consistency is crucial. Existing methods frequently encounter difficulties maintaining facial coherence due to misalignments between facial landmarks extracted from driving videos that provide head pose and expression cues and the facial geometry of the reference images. To address this limitation, we introduce the Facial Landmarks Transformation (FLT) method, which leverages a 3D Morphable Model to address this limitation. FLT converts 2D landmarks into a 3D face model, adjusts the 3D face model to align with the reference identity, and then transforms them back into 2D landmarks to guide the image-to-video generation process. This approach ensures accurate alignment with the reference facial geometry, enhancing the consistency between generated videos and reference images. Experimental results demonstrate that FLT effectively preserves facial identity, significantly improving pose-guided character animation models.