Causal inference with dyadic data in randomized experiments

TL;DR

This paper introduces a new causal inference framework for dyadic data in randomized experiments, addressing interference issues and providing estimators with proven asymptotic properties, demonstrated through large-scale social media experiments.

Contribution

It develops a novel design-based causal inference method specifically for dyadic outcomes, including estimators and variance measures, with theoretical guarantees and practical validation.

Findings

Proved the central limit theorem for the estimators.

Demonstrated correction of interference bias in numerical experiments.

Validated the approach in a large-scale WeChat experiment.

Abstract

Estimating the treatment effect within network structures is a key focus in online controlled experiments, particularly for social media platforms. We investigate a scenario where the unit-level outcome of interest comprises a series of dyadic outcomes, which is pervasive in many social network sources, spanning from microscale point-to-point messaging to macroscale international trades. Dyadic outcomes are of particular interest in online controlled experiments, capturing pairwise interactions as basic units for analysis. The dyadic nature of the data induces interference, as treatment assigned to one unit may affect outcomes involving connected pairs. We propose a novel design-based causal inference framework for dyadic outcomes in randomized experiments, develop estimators of the global average causal effect, and establish their asymptotic properties under different randomization designs. We prove the central limit theorem for the estimators and propose variance estimators to quantify the estimation uncertainty. The advantages of integrating dyadic data in randomized experiments are manifested in a variety of numerical experiments, especially in correcting interference bias. We implement our proposed method in a large-scale experiment on WeChat Channels, assessing the impact of a recommendation algorithm on users' interaction metrics.