LayeringDiff: Layered Image Synthesis via Generation, then Disassembly with Generative Knowledge

TL;DR

LayeringDiff introduces a novel layered image synthesis pipeline that generates a composite image then disassembles it into layers using pretrained models, enabling diverse and detailed layered image creation without extensive training.

Contribution

The paper presents LayeringDiff, a new method that synthesizes layered images by generating a composite and then disassembling it, leveraging pretrained models to avoid large-scale training.

Findings

Effective layered image synthesis demonstrated through experiments

Supports diverse content and object scales in layers

Refines layer details with high-frequency alignment modules

Abstract

Layers have become indispensable tools for professional artists, allowing them to build a hierarchical structure that enables independent control over individual visual elements. In this paper, we propose LayeringDiff, a novel pipeline for the synthesis of layered images, which begins by generating a composite image using an off-the-shelf image generative model, followed by disassembling the image into its constituent foreground and background layers. By extracting layers from a composite image, rather than generating them from scratch, LayeringDiff bypasses the need for large-scale training to develop generative capabilities for individual layers. Furthermore, by utilizing a pretrained off-the-shelf generative model, our method can produce diverse contents and object scales in synthesized layers. For effective layer decomposition, we adapt a large-scale pretrained generative prior to estimate foreground and background layers. We also propose high-frequency alignment modules to refine the fine-details of the estimated layers. Our comprehensive experiments demonstrate that our approach effectively synthesizes layered images and supports various practical applications.