Balancing Approach for Causal Inference at Scale

TL;DR

This paper introduces scalable algorithms for covariate balancing in causal inference, enabling efficient analysis on large datasets, demonstrated through applications at Snap Inc.

Contribution

It presents linear-time algorithms, DistEB and DistMS, for entropy balancing and MicroSynth, suitable for distributed frameworks, improving scalability for large-scale causal inference.

Findings

Bias and variance decrease with larger sample sizes

Algorithms are efficient and scalable for datasets with up to 1 million units

Application to real-world data at Snap Inc. validates effectiveness

Abstract

With the modern software and online platforms to collect massive amount of data, there is an increasing demand of applying causal inference methods at large scale when randomized experimentation is not viable. Weighting methods that directly incorporate covariate balancing have recently gained popularity for estimating causal effects in observational studies. These methods reduce the manual efforts required by researchers to iterate between propensity score modeling and balance checking until a satisfied covariate balance result. However, conventional solvers for determining weights lack the scalability to apply such methods on large scale datasets in companies like Snap Inc. To address the limitations and improve computational efficiency, in this paper we present scalable algorithms, DistEB and DistMS, for two balancing approaches: entropy balancing and MicroSynth. The solvers have linear time complexity and can be conveniently implemented in distributed computing frameworks such as Spark, Hive, etc. We study the properties of balancing approaches at different scales up to 1 million treated units and 487 covariates. We find that with larger sample size, both bias and variance in the causal effect estimation are significantly reduced. The results emphasize the importance of applying balancing approaches on large scale datasets. We combine the balancing approach with a synthetic control framework and deploy an end-to-end system for causal impact estimation at Snap Inc.