TalkPhoto: A Versatile Training-Free Conversational Assistant for Intelligent Image Editing

TL;DR

TalkPhoto is a training-free, conversational image editing framework that leverages large language models and existing editing methods to enable precise, flexible, and high-quality image manipulation through dialogue.

Contribution

It introduces a novel training-free approach that hierarchically invokes existing editing methods based on LLM analysis, eliminating the need for multi-instruction datasets.

Findings

More accurate invocation with fewer tokens

Higher editing quality across tasks

Stable and flexible integration of editing methods

Abstract

Thanks to the powerful language comprehension capabilities of Large Language Models (LLMs), existing instruction-based image editing methods have introduced Multimodal Large Language Models (MLLMs) to promote information exchange between instructions and images, ensuring the controllability and flexibility of image editing. However, these frameworks often build a multi-instruction dataset to train the model to handle multiple editing tasks, which is not only time-consuming and labor-intensive but also fails to achieve satisfactory results. In this paper, we present TalkPhoto, a versatile training-free image editing framework that facilitates precise image manipulation through conversational interaction. We instruct the open-source LLM with a specially designed prompt template to analyze user needs after receiving instructions and hierarchically invoke existing advanced editing methods, all without additional training. Moreover, we implement a plug-and-play and efficient invocation of image editing methods, allowing complex and unseen editing tasks to be integrated into the current framework, achieving stable and high-quality editing results. Extensive experiments demonstrate that our method not only provides more accurate invocation with fewer token consumption but also achieves higher editing quality across various image editing tasks.