Robust Causal Inference for Incremental Return on Ad Spend with Randomized Paired Geo Experiments

TL;DR

This paper introduces a robust, distribution-free estimator called 'Trimmed Match' for accurately estimating the incremental return on ad spend (iROAS) from randomized geo experiments, addressing heterogeneity and budget constraints.

Contribution

It develops a novel statistical framework for iROAS inference from paired geo experiments and proposes the 'Trimmed Match' estimator with a data-driven tuning parameter.

Findings

'Trimmed Match' outperforms alternative estimators in simulations.

The method is robust to violations of assumptions.

Real case studies demonstrate practical utility.

Abstract

Evaluating the incremental return on ad spend (iROAS) of a prospective online marketing strategy (i.e., the ratio of the strategy's causal effect on some response metric of interest relative to its causal effect on the ad spend) has become increasingly more important. Although randomized ``geo experiments'' are frequently employed for this evaluation, obtaining reliable estimates of iROAS can be challenging as oftentimes only a small number of highly heterogeneous units are used. Moreover, advertisers frequently impose budget constraints on their ad spends, which further complicates causal inference by introducing interference between the experimental units. In this paper, we formulate a novel statistical framework for inferring the iROAS of online advertising from randomized paired geo experiment which further motivates and provides new insights into Rosenbaum's arguments on instrumental variables, and we propose and develop a robust, distribution-free and interpretable estimator ``Trimmed Match'', as well as a data-driven choice of the tuning parameter which may be of independent interest. We investigate the sensitivity of Trimmed Match to some violations of its assumptions and show that it can be more efficient than some alternative estimators based on simulated data. We then demonstrate its practical utility with real case studies.