FacEnhance: Facial Expression Enhancing with Recurrent DDPMs

TL;DR

FacEnhance is a diffusion-based method that improves low-resolution facial expression videos by increasing resolution, adding background details, and maintaining identity, thus advancing resource-efficient high-quality facial expression synthesis.

Contribution

Introduces FacEnhance, a novel diffusion model that enhances low-resolution facial expression videos to high resolution with background details, combining efficiency and quality.

Findings

Achieves high-quality enhancement from 64x64 to 192x192 pixels.

Preserves identity and content consistency in generated videos.

Outperforms existing methods on the MUG database.

Abstract

Facial expressions, vital in non-verbal human communication, have found applications in various computer vision fields like virtual reality, gaming, and emotional AI assistants. Despite advancements, many facial expression generation models encounter challenges such as low resolution (e.g., 32x32 or 64x64 pixels), poor quality, and the absence of background details. In this paper, we introduce FacEnhance, a novel diffusion-based approach addressing constraints in existing low-resolution facial expression generation models. FacEnhance enhances low-resolution facial expression videos (64x64 pixels) to higher resolutions (192x192 pixels), incorporating background details and improving overall quality. Leveraging conditional denoising within a diffusion framework, guided by a background-free low-resolution video and a single neutral expression high-resolution image, FacEnhance generates a video incorporating the facial expression from the low-resolution video performed by the individual with background from the neutral image. By complementing lightweight low-resolution models, FacEnhance strikes a balance between computational efficiency and desirable image resolution and quality. Extensive experiments on the MUG facial expression database demonstrate the efficacy of FacEnhance in enhancing low-resolution model outputs to state-of-the-art quality while preserving content and identity consistency. FacEnhance represents significant progress towards resource-efficient, high-fidelity facial expression generation, Renewing outdated low-resolution methods to up-to-date standards.