Design-based inference for generalized causal effects in randomized experiments

TL;DR

This paper develops a unified, design-based inference framework for generalized causal effects in randomized experiments, addressing challenges in variance estimation and covariate adjustment for complex estimands.

Contribution

It introduces a model-assisted approach for regression adjustment of generalized causal effects, establishing consistency and asymptotic normality under misspecification.

Findings

Complete two-way cluster-robust variance estimator is consistent.

Covariate adjustment preserves consistency but not efficiency for nonlinear contrasts.

Standard heteroskedasticity-robust variance estimators are generally inconsistent.

Abstract

Generalized causal effect estimands, including the Mann-Whitney parameter and causal net benefit, provide flexible summaries of treatment effects in randomized experiments with non-Gaussian or multivariate outcomes. We develop a unified design-based inference framework for regression adjustment and variance estimation of a broad class of generalized causal effect estimands defined through pairwise contrast functions. Leveraging the theory of U-statistics and finite-population asymptotics, we establish the consistency and asymptotic normality of regression estimators constructed from individual pairs and per-unit pair averages, even when the working models are misspecified. Consequently, these estimators are model-assisted rather than model-based. In contrast to classical average treatment effect estimands, we show that for nonlinear contrast functions, covariate adjustment preserves consistency but does not admit a universal efficiency guarantee. For inference, we demonstrate that standard heteroskedasticity-robust and cluster-robust variance estimators are generally inconsistent in this setting. As a remedy, we prove that a complete two-way cluster-robust variance estimator, which fully accounts for pairwise dependence and reverse comparisons, is consistent.