SERP Interference Network and Its Applications in Search Advertising

TL;DR

This paper introduces a novel network-based approach to handle the challenges of A/B testing in search engine marketing, especially with anonymized user data, by constructing SERP interference networks for better experimental design.

Contribution

It proposes a new method leveraging censored observational data to build SERP interference networks and demonstrates its application in evaluating bidding algorithms.

Findings

Effective clustering of search query-product interactions

Successful application in bidding algorithm evaluation

Framework adaptable to anonymized user data

Abstract

Search Engine marketing teams in the e-commerce industry manage global search engine traffic to their websites with the aim to optimize long-term profitability by delivering the best possible customer experience on Search Engine Results Pages (SERPs). In order to do so, they need to run continuous and rapid Search Marketing A/B tests to continuously evolve and improve their products. However, unlike typical e-commerce A/B tests that can randomize based on customer identification, their tests face the challenge of anonymized users on search engines. On the other hand, simply randomizing on products violates Stable Unit Treatment Value Assumption for most treatments of interest. In this work, we propose leveraging censored observational data to construct bipartite (Search Query to Product Ad or Text Ad) SERP interference networks. Using a novel weighting function, we create weighted projections to form unipartite graphs which can then be use to create clusters to randomized on. We demonstrate this experimental design's application in evaluating a new bidding algorithm for Paid Search. Additionally, we provide a blueprint of a novel system architecture utilizing SageMaker which enables polyglot programming to implement each component of the experimental framework.