MoFRR: Mixture of Diffusion Models for Face Retouching Restoration

TL;DR

This paper introduces MoFRR, a novel mixture of diffusion models designed to restore original faces from retouched images, addressing complex retouching variations with specialized experts and demonstrating effectiveness on a new dataset.

Contribution

The paper proposes a new face retouching restoration task and a mixture of diffusion models with specialized experts to handle diverse retouching types.

Findings

MoFRR effectively restores original faces from retouched images.

The model outperforms existing methods on the RetouchingFFHQ++ dataset.

Specialized experts improve restoration accuracy across different retouching types.

Abstract

The widespread use of face retouching on social media platforms raises concerns about the authenticity of face images. While existing methods focus on detecting face retouching, how to accurately recover the original faces from the retouched ones has yet to be answered. This paper introduces Face Retouching Restoration (FRR), a novel computer vision task aimed at restoring original faces from their retouched counterparts. FRR differs from traditional image restoration tasks by addressing the complex retouching operations with various types and degrees, which focuses more on the restoration of the low-frequency information of the faces. To tackle this challenge, we propose MoFRR, Mixture of Diffusion Models for FRR. Inspired by DeepSeek's expert isolation strategy, the MoFRR uses sparse activation of specialized experts handling distinct retouching types and the engagement of a shared expert dealing with universal retouching traces. Each specialized expert follows a dual-branch structure with a DDIM-based low-frequency branch guided by an Iterative Distortion Evaluation Module (IDEM) and a Cross-Attention-based High-Frequency branch (HFCAM) for detail refinement. Extensive experiments on a newly constructed face retouching dataset, RetouchingFFHQ++, demonstrate the effectiveness of MoFRR for FRR.