Middleman Bias in Advertising: Aligning Relevance of Keyphrase Recommendations with Search

TL;DR

This paper investigates the bias introduced by search relevance systems in e-commerce advertising, proposing methods to better align keyphrase recommendations with actual search relevance to improve seller satisfaction.

Contribution

It introduces a new perspective on relevance bias as a middleman bias and compares different encoding models to improve keyphrase relevance alignment.

Findings

Search relevance systems can introduce bias affecting keyphrase relevance.

Cross encoders outperform bi-encoders in modeling relevance alignment.

Scalable solutions for eBay sellers are discussed.

Abstract

E-commerce sellers are recommended keyphrases based on their inventory on which they advertise to increase buyer engagement (clicks/sales). Keyphrases must be pertinent to items; otherwise, it can result in seller dissatisfaction and poor targeting -- towards that end relevance filters are employed. In this work, we describe the shortcomings of training relevance filter models on biased click/sales signals. We re-conceptualize advertiser keyphrase relevance as interaction between two dynamical systems -- Advertising which produces the keyphrases and Search which acts as a middleman to reach buyers. We discuss the bias of search relevance systems (middleman bias) and the need to align advertiser keyphrases with search relevance signals. We also compare the performance of cross encoders and bi-encoders in modeling this alignment and the scalability of such a solution for sellers at eBay.