Statistical Power for Estimating Treatment Effects Using Difference-in-Differences and Comparative Interrupted Time Series Designs with Variation in Treatment Timing

TL;DR

This paper develops new variance formulas for power analysis in difference-in-differences and comparative interrupted time series designs, explicitly incorporating treatment timing variation and other practical features, aiding in sample size determination.

Contribution

It introduces closed-form variance expressions for power analysis that include treatment timing variation, autocorrelation, and clustering, enhancing practical applicability of these methods.

Findings

Accounting for treatment timing increases required sample sizes.

DID estimators have more power than CITS and ITS estimators.

The Shiny R dashboard facilitates sample size calculations.

Abstract

This article develops new closed-form variance expressions for power analyses for commonly used difference-in-differences (DID) and comparative interrupted time series (CITS) panel data estimators. The main contribution is to incorporate variation in treatment timing into the analysis. The power formulas also account for other key design features that arise in practice: autocorrelated errors, unequal measurement intervals, and clustering due to the unit of treatment assignment. We consider power formulas for both cross-sectional and longitudinal models and allow for covariates. An illustrative power analysis provides guidance on appropriate sample sizes. The key finding is that accounting for treatment timing increases required sample sizes. Further, DID estimators have considerably more power than standard CITS and ITS estimators. An available Shiny R dashboard performs the sample size calculations for the considered estimators.