TextureDiffusion: Target Prompt Disentangled Editing for Various Texture Transfer

TL;DR

TextureDiffusion is a novel, tuning-free image editing method that enables disentangled and high-quality transfer of complex textures like clouds or fire, while preserving image structure and background.

Contribution

It introduces a new approach that sets the target prompt to "<texture>" for disentangled texture transfer, improving upon previous methods limited to simple textures.

Findings

Effective transfer of complex textures such as clouds and fire

Preserves input image structure and background well

Achieves harmonious texture blending in experiments

Abstract

Recently, text-guided image editing has achieved significant success. However, existing methods can only apply simple textures like wood or gold when changing the texture of an object. Complex textures such as cloud or fire pose a challenge. This limitation stems from that the target prompt needs to contain both the input image content and <texture>, restricting the texture representation. In this paper, we propose TextureDiffusion, a tuning-free image editing method applied to various texture transfer. Initially, the target prompt is directly set to "<texture>", making the texture disentangled from the input image content to enhance texture representation. Subsequently, query features in self-attention and features in residual blocks are utilized to preserve the structure of the input image. Finally, to maintain the background, we introduce an edit localization technique which blends the self-attention results and the intermediate latents. Comprehensive experiments demonstrate that TextureDiffusion can harmoniously transfer various textures with excellent structure and background preservation. Code is publicly available at https://github.com/THU-CVML/TextureDiffusion