Generative Modeling with Multi-Instance Reward Learning for E-commerce Creative Optimization

TL;DR

This paper introduces GenCO, a novel framework combining generative modeling and multi-instance reward learning to optimize creative combinations in e-commerce advertising, significantly boosting revenue.

Contribution

The paper presents a new two-stage architecture that efficiently generates and refines creative combinations using reinforcement learning and multi-instance reward attribution, addressing the large search space challenge.

Findings

Increased advertising revenue on a leading e-commerce platform.

Effective exploration and refinement of creative combinations.

Release of a large-scale industrial dataset for research.

Abstract

In e-commerce advertising, selecting the most compelling combination of creative elements -- such as titles, images, and highlights -- is critical for capturing user attention and driving conversions. However, existing methods often evaluate creative components individually, failing to navigate the exponentially large search space of possible combinations. To address this challenge, we propose a novel framework named GenCO that integrates generative modeling with multi-instance reward learning. Our unified two-stage architecture first employs a generative model to efficiently produce a diverse set of creative combinations. This generative process is optimized with reinforcement learning, enabling the model to effectively explore and refine its selections. Next, to overcome the challenge of sparse user feedback, a multi-instance learning model attributes combination-level rewards, such as clicks, to the individual creative elements. This allows the reward model to provide a more accurate feedback signal, which in turn guides the generative model toward creating more effective combinations. Deployed on a leading e-commerce platform, our approach has significantly increased advertising revenue, demonstrating its practical value. Additionally, we are releasing a large-scale industrial dataset to facilitate further research in this important domain.