Eliminating Contextual Prior Bias for Semantic Image Editing via Dual-Cycle Diffusion

TL;DR

This paper introduces Dual-Cycle Diffusion, a novel method to eliminate contextual prior bias in semantic image editing, ensuring more accurate modifications by generating unbiased masks through a bias elimination cycle.

Contribution

The paper proposes a dual-cycle diffusion framework with a bias elimination cycle and structural consistency cycle to reduce bias and improve editing accuracy in diffusion-based image editing.

Findings

Significant improvement in D-CLIP score from 0.272 to 0.283.

Effective bias removal demonstrated through experiments.

Code implementation available online.

Abstract

The recent success of text-to-image generation diffusion models has also revolutionized semantic image editing, enabling the manipulation of images based on query/target texts. Despite these advancements, a significant challenge lies in the potential introduction of contextual prior bias in pre-trained models during image editing, e.g., making unexpected modifications to inappropriate regions. To address this issue, we present a novel approach called Dual-Cycle Diffusion, which generates an unbiased mask to guide image editing. The proposed model incorporates a Bias Elimination Cycle that consists of both a forward path and an inverted path, each featuring a Structural Consistency Cycle to ensure the preservation of image content during the editing process. The forward path utilizes the pre-trained model to produce the edited image, while the inverted path converts the result back to the source image. The unbiased mask is generated by comparing differences between the processed source image and the edited image to ensure that both conform to the same distribution. Our experiments demonstrate the effectiveness of the proposed method, as it significantly improves the D-CLIP score from 0.272 to 0.283. The code will be available at https://github.com/JohnDreamer/DualCycleDiffsion.