Generative Image Layer Decomposition with Visual Effects

TL;DR

This paper introduces LayerDecomp, a generative framework that accurately decomposes images into layers with visual effects, enabling precise editing and object removal, trained on a novel dataset with simulated and real images.

Contribution

The paper presents a new generative model for image layer decomposition that preserves visual effects, along with a dataset pipeline and a consistency loss for improved training.

Findings

Outperforms existing methods in layer decomposition quality.

Enhances object removal and spatial editing tasks.

Achieves superior results across benchmarks and user studies.

Abstract

Recent advancements in large generative models, particularly diffusion-based methods, have significantly enhanced the capabilities of image editing. However, achieving precise control over image composition tasks remains a challenge. Layered representations, which allow for independent editing of image components, are essential for user-driven content creation, yet existing approaches often struggle to decompose image into plausible layers with accurately retained transparent visual effects such as shadows and reflections. We propose $\textbf{LayerDecomp}$, a generative framework for image layer decomposition which outputs photorealistic clean backgrounds and high-quality transparent foregrounds with faithfully preserved visual effects. To enable effective training, we first introduce a dataset preparation pipeline that automatically scales up simulated multi-layer data with synthesized visual effects. To further enhance real-world applicability, we supplement this simulated dataset with camera-captured images containing natural visual effects. Additionally, we propose a consistency loss which enforces the model to learn accurate representations for the transparent foreground layer when ground-truth annotations are not available. Our method achieves superior quality in layer decomposition, outperforming existing approaches in object removal and spatial editing tasks across several benchmarks and multiple user studies, unlocking various creative possibilities for layer-wise image editing. The project page is https://rayjryang.github.io/LayerDecomp.