Valid and Unobtrusive Measurement of Returns to Advertising through Asymmetric Budget Split

TL;DR

This paper introduces Asymmetric Budget Split, a novel, unobtrusive method for accurately measuring advertising returns by creating small budget asymmetries across comparable user groups, demonstrated on LinkedIn's Jobs Marketplace.

Contribution

The paper presents a new methodology for valid ad return measurement that is cost-effective and does not disrupt existing ad performance or require holdout groups.

Findings

Successfully deployed on LinkedIn's Jobs Marketplace

Provides valid measurement of incremental job applicants

Cost-effective and unobtrusive in practice

Abstract

Ad platforms require reliable measurement of advertising returns: what increase in performance (such as clicks or conversions) can an advertiser expect in return for additional budget on the platform? Even from the perspective of the platform, accurately measuring advertising returns is hard. Selection and omitted variable biases make estimates from observational methods unreliable, and straightforward experimentation is often costly or infeasible. We introduce Asymmetric Budget Split, a novel methodology for valid measurement of ad returns from the perspective of the platform. Asymmetric budget split creates small asymmetries in ad budget allocation across comparable partitions of the platform's userbase. By observing performance of the same ad at different budget levels while holding all other factors constant, the platform can obtain a valid measure of ad returns. The methodology is unobtrusive and cost-effective in that it does not require holdout groups or sacrifices in ad or marketplace performance. We discuss a successful deployment of asymmetric budget split to LinkedIn's Jobs Marketplace, an ad marketplace where it is used to measure returns from promotion budgets in terms of incremental job applicants. We outline operational considerations for practitioners and discuss further use cases such as budget-aware performance forecasting.