# Parallel Experimentation and Competitive Interference on Online Advertising Platforms

## TL;DR

This paper develops a framework to measure advertising effects during parallel experimentation on online platforms, revealing that ignoring competitive interference can significantly misestimate advertising impact, which affects decision-making.

## Contribution

It introduces a novel experimental design and estimators to accurately measure advertising effects amidst competitive interference on online platforms.

## Key findings

- Ignoring competitive interference can double the estimated advertising lift.
- The proposed framework improves the accuracy of advertising effect estimates.
- Application on JD.com demonstrates significant impact of competitive actions on measurement.

## Abstract

This paper studies the measurement of advertising effects on online platforms when parallel experimentation occurs, that is, when multiple advertisers experiment concurrently. It provides a framework that makes precise how parallel experimentation affects the experiment's value: while ignoring parallel experimentation yields an estimate of the average effect of advertising in-place, which has limited value in decision-making in an environment with variable advertising competition, accounting for parallel experimentation captures the actual uncertainty advertisers face due to competitive actions. It then implements an experimental design that enables the estimation of these effects on JD.com, a large e-commerce platform that is also a publisher of digital ads. Using traditional and kernel-based estimators, it shows that not accounting for competitive actions can result in the advertiser inaccurately estimating the advertising lift by a factor of two or higher, which can be consequential for decision-making.