TSI: an Ad Text Strength Indicator using Text-to-CTR and Semantic-Ad-Similarity

TL;DR

This paper introduces TSI, a novel ad text strength indicator that predicts CTR, finds similar high-performing ads, and offers improvement suggestions, enhancing ad effectiveness especially for small businesses with limited advertising experience.

Contribution

We propose a BERT-based CTR prediction model and a semantic similarity model for ad retrieval, enabling real-time ad text strength assessment and suggestions with high accuracy.

Findings

BERT-based CTR model improves cold start CTR prediction accuracy.

Semantic similarity model achieves 0.93 precision@1 for ad retrieval.

Online deployment shows sub-second latency and positive impact on ad performance.

Abstract

Coming up with effective ad text is a time consuming process, and particularly challenging for small businesses with limited advertising experience. When an inexperienced advertiser onboards with a poorly written ad text, the ad platform has the opportunity to detect low performing ad text, and provide improvement suggestions. To realize this opportunity, we propose an ad text strength indicator (TSI) which: (i) predicts the click-through-rate (CTR) for an input ad text, (ii) fetches similar existing ads to create a neighborhood around the input ad, (iii) and compares the predicted CTRs in the neighborhood to declare whether the input ad is strong or weak. In addition, as suggestions for ad text improvement, TSI shows anonymized versions of superior ads (higher predicted CTR) in the neighborhood. For (i), we propose a BERT based text-to-CTR model trained on impressions and clicks associated with an ad text. For (ii), we propose a sentence-BERT based semantic-ad-similarity model trained using weak labels from ad campaign setup data. Offline experiments demonstrate that our BERT based text-to-CTR model achieves a significant lift in CTR prediction AUC for cold start (new) advertisers compared to bag-of-words based baselines. In addition, our semantic-textual-similarity model for similar ads retrieval achieves a precision@1 of 0.93 (for retrieving ads from the same product category); this is significantly higher compared to unsupervised TF-IDF, word2vec, and sentence-BERT baselines. Finally, we share promising online results from advertisers in the Yahoo (Verizon Media) ad platform where a variant of TSI was implemented with sub-second end-to-end latency.