Towards Reliable Advertising Image Generation Using Human Feedback

TL;DR

This paper proposes a novel feedback-driven framework for improving the reliability and efficiency of advertising image generation using human feedback and a new dataset, leading to higher quality outputs with fewer attempts.

Contribution

It introduces RFNet for automatic image inspection, a recurrent generation process, and a regularization technique for diffusion models, along with the RF1M dataset for training and evaluation.

Findings

Increased rate of available generated images.

Reduced number of attempts needed in image generation.

Enhanced production efficiency without compromising image quality.

Abstract

In the e-commerce realm, compelling advertising images are pivotal for attracting customer attention. While generative models automate image generation, they often produce substandard images that may mislead customers and require significant labor costs to inspect. This paper delves into increasing the rate of available generated images. We first introduce a multi-modal Reliable Feedback Network (RFNet) to automatically inspect the generated images. Combining the RFNet into a recurrent process, Recurrent Generation, results in a higher number of available advertising images. To further enhance production efficiency, we fine-tune diffusion models with an innovative Consistent Condition regularization utilizing the feedback from RFNet (RFFT). This results in a remarkable increase in the available rate of generated images, reducing the number of attempts in Recurrent Generation, and providing a highly efficient production process without sacrificing visual appeal. We also construct a Reliable Feedback 1 Million (RF1M) dataset which comprises over one million generated advertising images annotated by human, which helps to train RFNet to accurately assess the availability of generated images and faithfully reflect the human feedback. Generally speaking, our approach offers a reliable solution for advertising image generation.