DreamPainter: Image Background Inpainting for E-commerce Scenarios

TL;DR

DreamPainter is a novel framework for e-commerce background inpainting that combines text prompts and reference images, achieving high product consistency and realistic backgrounds in challenging scenarios.

Contribution

The paper introduces DreamPainter, a new method that effectively integrates text and visual references for improved e-commerce image background inpainting.

Findings

Outperforms state-of-the-art in product-background consistency

Successfully integrates text and reference images for flexible control

Demonstrates high-quality, realistic background generation in experiments

Abstract

Although diffusion-based image genenation has been widely explored and applied, background generation tasks in e-commerce scenarios still face significant challenges. The first challenge is to ensure that the generated products are consistent with the given product inputs while maintaining a reasonable spatial arrangement, harmonious shadows, and reflections between foreground products and backgrounds. Existing inpainting methods fail to address this due to the lack of domain-specific data. The second challenge involves the limitation of relying solely on text prompts for image control, as effective integrating visual information to achieve precise control in inpainting tasks remains underexplored. To address these challenges, we introduce DreamEcom-400K, a high-quality e-commerce dataset containing accurate product instance masks, background reference images, text prompts, and aesthetically pleasing product images. Based on this dataset, we propose DreamPainter, a novel framework that not only utilizes text prompts for control but also flexibly incorporates reference image information as an additional control signal. Extensive experiments demonstrate that our approach significantly outperforms state-of-the-art methods, maintaining high product consistency while effectively integrating both text prompt and reference image information.