FixTalk: Taming Identity Leakage for High-Quality Talking Head Generation in Extreme Cases

TL;DR

FixTalk is a new framework that improves talking head generation by reducing identity leakage and artifacts, using novel indicators to decouple identity from motion and enhance details for higher quality results.

Contribution

The paper introduces FixTalk, a framework with EMI and EDI to address identity leakage and rendering artifacts in talking head generation, a novel approach in this domain.

Findings

Effective mitigation of identity leakage and artifacts

Superior performance over state-of-the-art methods

High-quality talking head generation in extreme cases

Abstract

Talking head generation is gaining significant importance across various domains, with a growing demand for high-quality rendering. However, existing methods often suffer from identity leakage (IL) and rendering artifacts (RA), particularly in extreme cases. Through an in-depth analysis of previous approaches, we identify two key insights: (1) IL arises from identity information embedded within motion features, and (2) this identity information can be leveraged to address RA. Building on these findings, this paper introduces FixTalk, a novel framework designed to simultaneously resolve both issues for high-quality talking head generation. Firstly, we propose an Enhanced Motion Indicator (EMI) to effectively decouple identity information from motion features, mitigating the impact of IL on generated talking heads. To address RA, we introduce an Enhanced Detail Indicator (EDI), which utilizes the leaked identity information to supplement missing details, thus fixing the artifacts. Extensive experiments demonstrate that FixTalk effectively mitigates IL and RA, achieving superior performance compared to state-of-the-art methods.