Policy Evaluation for Temporal and/or Spatial Dependent Experiments

TL;DR

This paper introduces a novel Varying Coefficient Decision Process model to evaluate causal effects of policies in complex temporal and spatial experiments, enabling detailed analysis of direct and indirect effects.

Contribution

It proposes a new VCDP model for causal inference in dependent experiments and provides estimation procedures for direct and indirect effects with statistical guarantees.

Findings

Effective in capturing evolving treatment effects

Accurate estimation of direct and indirect effects

Validated through simulations and real data

Abstract

The aim of this paper is to establish a causal link between the policies implemented by technology companies and the outcomes they yield within intricate temporal and/or spatial dependent experiments. We propose a novel temporal/spatio-temporal Varying Coefficient Decision Process (VCDP) model, capable of effectively capturing the evolving treatment effects in situations characterized by temporal and/or spatial dependence. Our methodology encompasses the decomposition of the Average Treatment Effect (ATE) into the Direct Effect (DE) and the Indirect Effect (IE). We subsequently devise comprehensive procedures for estimating and making inferences about both DE and IE. Additionally, we provide a rigorous analysis of the statistical properties of these procedures, such as asymptotic power. To substantiate the effectiveness of our approach, we carry out extensive simulations and real data analyses.