Optimized Covariance Design for AB Test on Social Network under Interference

TL;DR

This paper introduces an optimized covariance design for A/B testing on social networks that balances bias and variance under interference, improving estimation accuracy of treatment effects.

Contribution

It proposes a novel experimental design method that minimizes the MSE upper bound by optimizing the covariance matrix of treatment assignment, accounting for network interference.

Findings

Outperforms existing methods in simulation studies.

Effectively balances bias and variance under interference.

Demonstrates robustness to model misspecification.

Abstract

Online A/B tests have become increasingly popular and important for social platforms. However, accurately estimating the global average treatment effect (GATE) has proven to be challenging due to network interference, which violates the Stable Unit Treatment Value Assumption (SUTVA) and poses a great challenge to experimental design. Existing network experimental design research was mostly based on the unbiased Horvitz-Thompson (HT) estimator with substantial data trimming to ensure unbiasedness at the price of high resultant estimation variance. In this paper, we strive to balance the bias and variance in designing randomized network experiments. Under a potential outcome model with 1-hop interference, we derive the bias and variance of the standard HT estimator and reveal their relation to the network topological structure and the covariance of the treatment assignment vector. We then propose to formulate the experimental design problem to optimize the covariance matrix of the treatment assignment vector to achieve the bias and variance balance by minimizing a well-crafted upper bound of the mean squared error (MSE) of the estimator, which allows us to decouple the unknown interference effect component and the experimental design component. An efficient projected gradient descent algorithm is presented to implement the desired randomization scheme. Finally, we carry out extensive simulation studies 2 to demonstrate the advantages of our proposed method over other existing methods in many settings, with different levels of model misspecification.