DragText: Rethinking Text Embedding in Point-based Image Editing

TL;DR

This paper introduces DragText, a method that optimizes text embeddings during point-based image editing to improve content consistency and manipulation accuracy in diffusion models.

Contribution

It explores the interaction between text and image embeddings and proposes a novel optimization of text embeddings during editing, enhancing existing diffusion-based methods.

Findings

Improved content preservation during image editing.

Enhanced manipulation accuracy with optimized text embeddings.

Seamless integration with existing diffusion-based drag methods.

Abstract

Point-based image editing enables accurate and flexible control through content dragging. However, the role of text embedding during the editing process has not been thoroughly investigated. A significant aspect that remains unexplored is the interaction between text and image embeddings. During the progressive editing in a diffusion model, the text embedding remains constant. As the image embedding increasingly diverges from its initial state, the discrepancy between the image and text embeddings presents a significant challenge. In this study, we found that the text prompt significantly influences the dragging process, particularly in maintaining content integrity and achieving the desired manipulation. Upon these insights, we propose DragText, which optimizes text embedding in conjunction with the dragging process to pair with the modified image embedding. Simultaneously, we regularize the text optimization process to preserve the integrity of the original text prompt. Our approach can be seamlessly integrated with existing diffusion-based drag methods, enhancing performance with only a few lines of code.