A modified $\chi^2$-test for uplift models with applications in marketing performance measurement

TL;DR

This paper introduces a modified chi-squared test for comparing uplift in marketing campaigns, providing a statistically rigorous method with improved power demonstrated through simulations and real-life application.

Contribution

A new chi-squared statistic for testing differences in uplift between campaigns, with proven asymptotic distribution and enhanced decisiveness over existing methods.

Findings

The new test is asymptotically chi-squared distributed.

It shows higher power in simulations compared to alternative approaches.

Effective in real marketing data analysis.

Abstract

Uplift, essentially being the difference between two probabilities, is a central number in marketing performance measurement. A frequent question in applications is whether the uplifts of two campaigns are significantly different. In this article we present a new $\chi^2$-statistic which allows to answer this question by performing a statistical test. We show that this statistic is asymptotically $\chi^2$-distributed and demonstrate its application in a real life example. By running simulations with this new and alternative approaches, we find our suggested test to exhibit a better decisive power.