LAMS-Edit: Latent and Attention Mixing with Schedulers for Improved Content Preservation in Diffusion-Based Image and Style Editing

TL;DR

LAMS-Edit introduces a novel framework that combines latent representations and attention maps with schedulers during diffusion-based image editing, improving content preservation and editing precision in real-image and style transfer tasks.

Contribution

It proposes LAMS-Edit, a new method that leverages intermediate states and attention mixing with schedulers to enhance content preservation in diffusion-based image editing.

Findings

Effectively balances content preservation and edit application.

Supports region-specific editing and style transfer.

Demonstrates superior performance in experiments.

Abstract

Text-to-Image editing using diffusion models faces challenges in balancing content preservation with edit application and handling real-image editing. To address these, we propose LAMS-Edit, leveraging intermediate states from the inversion process--an essential step in real-image editing--during edited image generation. Specifically, latent representations and attention maps from both processes are combined at each step using weighted interpolation, controlled by a scheduler. This technique, Latent and Attention Mixing with Schedulers (LAMS), integrates with Prompt-to-Prompt (P2P) to form LAMS-Edit--an extensible framework that supports precise editing with region masks and enables style transfer via LoRA. Extensive experiments demonstrate that LAMS-Edit effectively balances content preservation and edit application.