Improving Editability in Image Generation with Layer-wise Memory

TL;DR

This paper introduces a layer-wise memory framework for image generation that enhances sequential editing capabilities, allowing for natural, coherent modifications with minimal user input and maintaining content consistency across multiple edits.

Contribution

The paper proposes a novel layer-wise memory approach and associated techniques to improve sequential image editing, addressing limitations of existing methods in maintaining content coherence and natural integration of new elements.

Findings

Superior performance in iterative editing tasks

Effective preservation of scene coherence across edits

Minimal user effort required for high-quality results

Abstract

Most real-world image editing tasks require multiple sequential edits to achieve desired results. Current editing approaches, primarily designed for single-object modifications, struggle with sequential editing: especially with maintaining previous edits along with adapting new objects naturally into the existing content. These limitations significantly hinder complex editing scenarios where multiple objects need to be modified while preserving their contextual relationships. We address this fundamental challenge through two key proposals: enabling rough mask inputs that preserve existing content while naturally integrating new elements and supporting consistent editing across multiple modifications. Our framework achieves this through layer-wise memory, which stores latent representations and prompt embeddings from previous edits. We propose Background Consistency Guidance that leverages memorized latents to maintain scene coherence and Multi-Query Disentanglement in cross-attention that ensures natural adaptation to existing content. To evaluate our method, we present a new benchmark dataset incorporating semantic alignment metrics and interactive editing scenarios. Through comprehensive experiments, we demonstrate superior performance in iterative image editing tasks with minimal user effort, requiring only rough masks while maintaining high-quality results throughout multiple editing steps.