Beyond Quality: Unlocking Diversity in Ad Headline Generation with Large Language Models

TL;DR

This paper introduces DIVER, a large language model framework that jointly optimizes for both diversity and quality in ad headline generation, leading to more engaging and varied advertising content.

Contribution

The paper presents a novel multi-stage optimization framework combining supervised fine-tuning and reinforcement learning for diverse and high-quality ad headlines.

Findings

DIVER effectively balances diversity and quality in ad headlines.

Deployment results show 4.0% improvement in advertiser value and 1.4% in CTR.

Framework scalable to large industrial datasets.

Abstract

The generation of ad headlines plays a vital role in modern advertising, where both quality and diversity are essential to engage a broad range of audience segments. Current approaches primarily optimize language models for headline quality or click-through rates (CTR), often overlooking the need for diversity and resulting in homogeneous outputs. To address this limitation, we propose DIVER, a novel framework based on large language models (LLMs) that are jointly optimized for both diversity and quality. We first design a semantic- and stylistic-aware data generation pipeline that automatically produces high-quality training pairs with ad content and multiple diverse headlines. To achieve the goal of generating high-quality and diversified ad headlines within a single forward pass, we propose a multi-stage multi-objective optimization framework with supervised fine-tuning (SFT) and reinforcement learning (RL). Experiments on real-world industrial datasets demonstrate that DIVER effectively balances quality and diversity. Deployed on a large-scale content-sharing platform serving hundreds of millions of users, our framework improves advertiser value (ADVV) and CTR by 4.0% and 1.4%.