A Simple Model to Estimate Sharing Effects in Social Networks

TL;DR

This paper introduces a simple MDP-based model for estimating sharing effects in social networks, addressing interference issues in A/B testing, and provides an unbiased estimator that outperforms existing methods in synthetic experiments.

Contribution

The paper proposes a novel Markov Decision Process model for user sharing behavior and derives an unbiased estimator for treatment effects in social networks.

Findings

Unbiased estimator outperforms existing methods in synthetic tests.

Model effectively captures sharing behavior and interference effects.

Provides a practical approach for causal inference in social network experiments.

Abstract

Randomised Controlled Trials (RCTs) are the gold standard for estimating treatment effects across many fields of science. Technology companies have adopted A/B-testing methods as a modern RCT counterpart, where end-users are randomly assigned various system variants and user behaviour is tracked continuously. The objective is then to estimate the causal effect that the treatment variant would have on certain metrics of interest to the business. When the outcomes for randomisation units -- end-users in this case -- are not statistically independent, this obfuscates identifiability of treatment effects, and harms decision-makers' observability of the system. Social networks exemplify this, as they are designed to promote inter-user interactions. This interference by design notoriously complicates measurement of, e.g., the effects of sharing. In this work, we propose a simple Markov Decision Process (MDP)-based model describing user sharing behaviour in social networks. We derive an unbiased estimator for treatment effects under this model, and demonstrate through reproducible synthetic experiments that it outperforms existing methods by a significant margin.