Striking Gold in Advertising: Standardization and Exploration of Ad Text Generation

TL;DR

This paper standardizes automatic ad text generation, introduces a benchmark dataset called CAMERA, and evaluates various models including large language models to understand current capabilities and challenges.

Contribution

It provides the first comprehensive benchmark dataset for ATG, standardizes the task, and evaluates multiple models including LLMs for industry-relevant ad text generation.

Findings

Nine baseline models evaluated, from classical to state-of-the-art.

Large language models show promising results but still face challenges.

Existing metrics partially align with human evaluations.

Abstract

In response to the limitations of manual ad creation, significant research has been conducted in the field of automatic ad text generation (ATG). However, the lack of comprehensive benchmarks and well-defined problem sets has made comparing different methods challenging. To tackle these challenges, we standardize the task of ATG and propose a first benchmark dataset, CAMERA, carefully designed and enabling the utilization of multi-modal information and facilitating industry-wise evaluations. Our extensive experiments with a variety of nine baselines, from classical methods to state-of-the-art models including large language models (LLMs), show the current state and the remaining challenges. We also explore how existing metrics in ATG and an LLM-based evaluator align with human evaluations.