Patch-enhanced Mask Encoder Prompt Image Generation

TL;DR

This paper introduces a patch-enhanced mask encoder method for AI-generated image content that improves product description accuracy and background diversity, outperforming existing techniques in visual quality and FID scores.

Contribution

It proposes a novel patch-enhanced mask encoder approach with three components to better control content and backgrounds in AI-generated images for advertising.

Findings

Achieves higher visual quality and FID scores than previous methods.

Ensures more accurate product descriptions in generated images.

Preserves diverse backgrounds effectively.

Abstract

Artificial Intelligence Generated Content(AIGC), known for its superior visual results, represents a promising mitigation method for high-cost advertising applications. Numerous approaches have been developed to manipulate generated content under different conditions. However, a crucial limitation lies in the accurate description of products in advertising applications. Applying previous methods directly may lead to considerable distortion and deformation of advertised products, primarily due to oversimplified content control conditions. Hence, in this work, we propose a patch-enhanced mask encoder approach to ensure accurate product descriptions while preserving diverse backgrounds. Our approach consists of three components Patch Flexible Visibility, Mask Encoder Prompt Adapter and an image Foundation Model. Patch Flexible Visibility is used for generating a more reasonable background image. Mask Encoder Prompt Adapter enables region-controlled fusion. We also conduct an analysis of the structure and operational mechanisms of the Generation Module. Experimental results show our method can achieve the highest visual results and FID scores compared with other methods.