DEGSTalk: Decomposed Per-Embedding Gaussian Fields for Hair-Preserving Talking Face Synthesis

TL;DR

DEGSTalk is a novel 3D Gaussian Splatting-based method that synthesizes realistic talking face videos with long hair, capturing fine facial features and hair dynamics more accurately than previous approaches.

Contribution

It introduces Deformable Pre-Embedding Gaussian Fields and a Dynamic Hair-Preserving Rendering technique for improved facial and hair realism in talking face synthesis.

Findings

Enhanced realism in facial expressions and hair motion.

Better handling of complex facial dynamics.

Improved synthesis quality over existing methods.

Abstract

Accurately synthesizing talking face videos and capturing fine facial features for individuals with long hair presents a significant challenge. To tackle these challenges in existing methods, we propose a decomposed per-embedding Gaussian fields (DEGSTalk), a 3D Gaussian Splatting (3DGS)-based talking face synthesis method for generating realistic talking faces with long hairs. Our DEGSTalk employs Deformable Pre-Embedding Gaussian Fields, which dynamically adjust pre-embedding Gaussian primitives using implicit expression coefficients. This enables precise capture of dynamic facial regions and subtle expressions. Additionally, we propose a Dynamic Hair-Preserving Portrait Rendering technique to enhance the realism of long hair motions in the synthesized videos. Results show that DEGSTalk achieves improved realism and synthesis quality compared to existing approaches, particularly in handling complex facial dynamics and hair preservation. Our code will be publicly available at https://github.com/CVI-SZU/DEGSTalk.