UIP2P: Unsupervised Instruction-based Image Editing via Edit Reversibility Constraint

TL;DR

This paper introduces UIP2P, an unsupervised instruction-based image editing method that uses an Edit Reversibility Constraint to enable training without ground-truth edited images, improving flexibility and reducing bias.

Contribution

The paper presents a novel Edit Reversibility Constraint that allows training on real image-caption data without ground-truth edits, advancing unsupervised instruction-based image editing.

Findings

Outperforms existing methods in edit fidelity and precision.

Operates effectively without ground-truth edited images.

Reduces biases from supervised datasets.

Abstract

We propose an unsupervised instruction-based image editing approach that removes the need for ground-truth edited images during training. Existing methods rely on supervised learning with triplets of input images, ground-truth edited images, and edit instructions. These triplets are typically generated either by existing editing methods, introducing biases, or through human annotations, which are costly and limit generalization. Our approach addresses these challenges by introducing a novel editing mechanism called Edit Reversibility Constraint (ERC), which applies forward and reverse edits in one training step and enforces alignment in image, text, and attention spaces. This allows us to bypass the need for ground-truth edited images and unlock training for the first time on datasets comprising either real image-caption pairs or image-caption-instruction triplets. We empirically show that our approach performs better across a broader range of edits with high-fidelity and precision. By eliminating the need for pre-existing datasets of triplets, reducing biases associated with current methods, and proposing ERC, our work represents a significant advancement in unblocking scaling of instruction-based image editing.