User-friendly Image Editing with Minimal Text Input: Leveraging Captioning and Injection Techniques

TL;DR

This paper introduces a user-friendly approach to text-driven image editing that reduces prompt engineering effort by leveraging captioning and injection techniques, making image editing more accessible and efficient.

Contribution

It proposes methods combining prompt generation frameworks to simplify prompt creation, improving user experience in text-driven image editing.

Findings

Prompt quality significantly impacts editing results.

Our method achieves results comparable to ground-truth prompts.

Simplified prompt generation enhances user-friendliness.

Abstract

Recent text-driven image editing in diffusion models has shown remarkable success. However, the existing methods assume that the user's description sufficiently grounds the contexts in the source image, such as objects, background, style, and their relations. This assumption is unsuitable for real-world applications because users have to manually engineer text prompts to find optimal descriptions for different images. From the users' standpoint, prompt engineering is a labor-intensive process, and users prefer to provide a target word for editing instead of a full sentence. To address this problem, we first demonstrate the importance of a detailed text description of the source image, by dividing prompts into three categories based on the level of semantic details. Then, we propose simple yet effective methods by combining prompt generation frameworks, thereby making the prompt engineering process more user-friendly. Extensive qualitative and quantitative experiments demonstrate the importance of prompts in text-driven image editing and our method is comparable to ground-truth prompts.