LLMDistill4Ads: Using Cross-Encoders to Distill from LLM Signals for Advertiser Keyphrase Recommendations at eBay

TL;DR

This paper proposes a distillation framework using cross-encoders and LLM signals to improve keyphrase recommendations for eBay ads, addressing bias and diversity issues.

Contribution

It introduces a novel distillation approach combining LLMs, cross-encoders, and bi-encoder models to enhance relevance and diversity in ad keyphrase suggestions.

Findings

Reduced click-induced biases in recommendations

Improved diversity of suggested keyphrases

Better alignment with seller, search, and buyer preferences

Abstract

E-commerce sellers are advised to bid on keyphrases to boost their advertising campaigns. These keyphrases must be relevant to prevent irrelevant items from cluttering Search systems and to maintain positive seller perception. It is vital that keyphrase suggestions align with seller, Search, and buyer judgments. Given the challenges in collecting negative feedback in these systems, LLMs have been used as a scalable proxy for human judgments. We present an empirical study on a major e-commerce platform of a distillation framework involving an LLM teacher, a cross-encoder assistant and a bi-encoder Embedding Based Retrieval (EBR) student model, aimed at mitigating click-induced biases and provide more diverse keyphrase recommendations while aligning advertising, search and buyer preferences.