RELATE: A Reinforcement Learning-Enhanced LLM Framework for Advertising Text Generation

TL;DR

RELATE is an end-to-end reinforcement learning framework that unifies ad text generation and performance optimization, leading to improved conversion rates and better alignment with advertiser goals in online advertising.

Contribution

It introduces a reinforcement learning-based unified model that integrates generation with performance and compliance objectives, enhancing ad effectiveness and efficiency.

Findings

Outperforms baseline models on large-scale datasets.

Achieves significant improvements in CTCVR in online deployment.

Effectively incorporates conversion metrics into the generation process.

Abstract

In online advertising, advertising text plays a critical role in attracting user engagement and driving advertiser value. Existing industrial systems typically follow a two-stage paradigm, where candidate texts are first generated and subsequently aligned with online performance metrics such as click-through rate(CTR). This separation often leads to misaligned optimization objectives and low funnel efficiency, limiting global optimality. To address these limitations, we propose RELATE, a reinforcement learning-based end-to-end framework that unifies generation and objective alignment within a single model. Instead of decoupling text generation from downstream metric alignment, RELATE integrates performance and compliance objectives directly into the generation process via policy learning. To better capture ultimate advertiser value beyond click-level signals, We incorporate conversion-oriented metrics into the objective and jointly model them with compliance constraints as multi-dimensional rewards, enabling the model to generate high-quality ad texts that improve conversion performance under policy constraints. Extensive experiments on large-scale industrial datasets demonstrate that RELATE consistently outperforms baselines. Furthermore, online deployment on a production advertising platform yields statistically significant improvements in click-through conversion rate(CTCVR) under strict policy constraints, validating the robustness and real-world effectiveness of the proposed framework.