Diff-Privacy: Diffusion-based Face Privacy Protection

TL;DR

This paper introduces Diff-Privacy, a diffusion-based method that unifies facial anonymization and visual identity hiding, effectively protecting privacy while maintaining recognition accuracy.

Contribution

It proposes a novel diffusion model framework with multi-scale image inversion and embedding strategies for combined face anonymization and identity hiding.

Findings

Effective facial privacy protection demonstrated through extensive experiments.

Unified approach successfully balances anonymization and recognition accuracy.

Diff-Privacy outperforms existing methods in privacy preservation tasks.

Abstract

Privacy protection has become a top priority as the proliferation of AI techniques has led to widespread collection and misuse of personal data. Anonymization and visual identity information hiding are two important facial privacy protection tasks that aim to remove identification characteristics from facial images at the human perception level. However, they have a significant difference in that the former aims to prevent the machine from recognizing correctly, while the latter needs to ensure the accuracy of machine recognition. Therefore, it is difficult to train a model to complete these two tasks simultaneously. In this paper, we unify the task of anonymization and visual identity information hiding and propose a novel face privacy protection method based on diffusion models, dubbed Diff-Privacy. Specifically, we train our proposed multi-scale image inversion module (MSI) to obtain a set of SDM format conditional embeddings of the original image. Based on the conditional embeddings, we design corresponding embedding scheduling strategies and construct different energy functions during the denoising process to achieve anonymization and visual identity information hiding. Extensive experiments have been conducted to validate the effectiveness of our proposed framework in protecting facial privacy.