FunEditor: Achieving Complex Image Edits via Function Aggregation with Diffusion Models

TL;DR

FunEditor is a diffusion model that enables complex image edits by aggregating simple functions, achieving faster inference and better results than existing methods, especially for tasks like object movement and pasting.

Contribution

It introduces a novel approach to image editing with diffusion models by learning atomic functions and aggregating them for complex edits, improving efficiency and accuracy.

Findings

Outperforms recent inference-time optimization and fine-tuned models.

Achieves 5-24x inference speedups with only 4 inference steps.

Successfully handles complex editing tasks like object movement and pasting.

Abstract

Diffusion models have demonstrated outstanding performance in generative tasks, making them ideal candidates for image editing. Recent studies highlight their ability to apply desired edits effectively by following textual instructions, yet with two key challenges remaining. First, these models struggle to apply multiple edits simultaneously, resulting in computational inefficiencies due to their reliance on sequential processing. Second, relying on textual prompts to determine the editing region can lead to unintended alterations to the image. We introduce FunEditor, an efficient diffusion model designed to learn atomic editing functions and perform complex edits by aggregating simpler functions. This approach enables complex editing tasks, such as object movement, by aggregating multiple functions and applying them simultaneously to specific areas. Our experiments demonstrate that FunEditor significantly outperforms recent inference-time optimization methods and fine-tuned models, either quantitatively across various metrics or through visual comparisons or both, on complex tasks like object movement and object pasting. In the meantime, with only 4 steps of inference, FunEditor achieves 5-24x inference speedups over existing popular methods. The code is available at: mhmdsmdi.github.io/funeditor/.