LinkLouvain: Link-Aware A/B Testing and Its Application on Online Marketing Campaign

TL;DR

This paper introduces LinkLouvain, a link-aware A/B testing method that accounts for user interaction interference in online marketing campaigns, providing more accurate treatment effect estimates through network analysis.

Contribution

The paper proposes a novel network A/B testing approach, LinkLouvain, that reduces interference effects and improves accuracy in estimating campaign impact.

Findings

LinkLouvain outperforms existing methods in real-world data.

The method is successfully deployed in actual marketing campaigns.

It provides more reliable ATE estimates by considering network interference.

Abstract

A lot of online marketing campaigns aim to promote user interaction. The average treatment effect (ATE) of campaign strategies need to be monitored throughout the campaign. A/B testing is usually conducted for such needs, whereas the existence of user interaction can introduce interference to normal A/B testing. With the help of link prediction, we design a network A/B testing method LinkLouvain to minimize graph interference and it gives an accurate and sound estimate of the campaign's ATE. In this paper, we analyze the network A/B testing problem under a real-world online marketing campaign, describe our proposed LinkLouvain method, and evaluate it on real-world data. Our method achieves significant performance compared with others and is deployed in the online marketing campaign.