Fuzzy-Conditioned Diffusion and Diffusion Projection Attention Applied to Facial Image Correction

TL;DR

This paper introduces a fuzzy-conditioned diffusion method that allows pixel-wise control for facial image correction, leveraging diffusion priors and attention maps for interpretable and autonomous editing.

Contribution

It presents a novel fuzzy-conditioning technique for diffusion models enabling pixel-wise control and applies it to facial correction using diffusion-derived attention maps.

Findings

Enables controllable, pixel-wise image modifications.

Provides interpretable and autonomous facial correction.

Demonstrates effective use of diffusion priors and attention maps.

Abstract

Image diffusion has recently shown remarkable performance in image synthesis and implicitly as an image prior. Such a prior has been used with conditioning to solve the inpainting problem, but only supporting binary user-based conditioning. We derive a fuzzy-conditioned diffusion, where implicit diffusion priors can be exploited with controllable strength. Our fuzzy conditioning can be applied pixel-wise, enabling the modification of different image components to varying degrees. Additionally, we propose an application to facial image correction, where we combine our fuzzy-conditioned diffusion with diffusion-derived attention maps. Our map estimates the degree of anomaly, and we obtain it by projecting on the diffusion space. We show how our approach also leads to interpretable and autonomous facial image correction.