DreamMix: Decoupling Object Attributes for Enhanced Editability in Customized Image Inpainting

TL;DR

DreamMix is a diffusion-based image inpainting framework that effectively separates object attributes from identity, enabling more controllable and editable image modifications while preserving original identities.

Contribution

It introduces a novel attribute decoupling mechanism, textual attribute substitution, and a disentangled inpainting framework to improve attribute editability and identity preservation.

Findings

Outperforms existing methods in attribute editing and object insertion tasks.

Achieves a better balance between identity preservation and attribute flexibility.

Demonstrates versatility across various inpainting applications.

Abstract

Subject-driven image inpainting has recently gained prominence in image editing with the rapid advancement of diffusion models. Beyond image guidance, recent studies have explored incorporating text guidance to achieve identity-preserved yet locally editable object inpainting. However, these methods still suffer from identity overfitting, where original attributes remain entangled with target textual instructions. To overcome this limitation, we propose DreamMix, a diffusion-based framework adept at inserting target objects into user-specified regions while concurrently enabling arbitrary text-driven attribute modifications. DreamMix introduces three key components: (i) an Attribute Decoupling Mechanism (ADM) that synthesizes diverse attribute-augmented image-text pairs to mitigate overfitting; (ii) a Textual Attribute Substitution (TAS) module that isolates target attributes via orthogonal decomposition, and (iii) a Disentangled Inpainting Framework (DIF) that seperates local generation from global harmonization. Extensive experiments across multiple inpainting backbones demonstrate that DreamMix achieves a superior balance between identity preservation and attribute editability across diverse applications, including object insertion, attribute editing, and small object inpainting.