E-Commerce Inpainting with Mask Guidance in Controlnet for Reducing Overcompletion

TL;DR

This paper presents a novel approach to e-commerce image inpainting that reduces overcompletion by using mask guidance and fine-tuned models, improving the accuracy of product background restoration.

Contribution

The paper introduces a train-free mask guidance method combined with an instance mask fine-tuned inpainting model to effectively reduce overcompletion in diffusion-based e-commerce image generation.

Findings

Effective reduction of overcompletion in product image inpainting

Improved background restoration matching main product features

Promising practical application results

Abstract

E-commerce image generation has always been one of the core demands in the e-commerce field. The goal is to restore the missing background that matches the main product given. In the post-AIGC era, diffusion models are primarily used to generate product images, achieving impressive results. This paper systematically analyzes and addresses a core pain point in diffusion model generation: overcompletion, which refers to the difficulty in maintaining product features. We propose two solutions: 1. Using an instance mask fine-tuned inpainting model to mitigate this phenomenon; 2. Adopting a train-free mask guidance approach, which incorporates refined product masks as constraints when combining ControlNet and UNet to generate the main product, thereby avoiding overcompletion of the product. Our method has achieved promising results in practical applications and we hope it can serve as an inspiring technical report in this field.