Analysis of Stepped-Wedge Cluster Randomized Trials when treatment effects vary by exposure time or calendar time

TL;DR

This paper investigates how different assumptions about treatment effect timing in stepped-wedge cluster randomized trials impact the accuracy of effect estimates, highlighting potential biases and the importance of correct model specification.

Contribution

It evaluates the robustness of various estimators under misspecified models for treatment effect variation by exposure and calendar time in SW-CRTs.

Findings

Immediate effect estimator is robust to calendar time variation.

Misspecification can lead to severely biased estimates, even reversing the sign.

Careful consideration of effect timing is crucial for valid analysis.

Abstract

Stepped-wedge cluster randomized trials (SW-CRTs) are traditionally analyzed with models that assume an immediate and sustained treatment effect. Previous work has shown that making such an assumption in the analysis of SW-CRTs when the true underlying treatment effect varies by exposure time can produce severely misleading estimates. Alternatively, the true underlying treatment effect might vary by calendar time. Comparatively less work has examined treatment effect structure misspecification in this setting. Here, we evaluate the behavior of the mixed effects model-based immediate treatment effect, exposure time-averaged treatment effect, and calendar time-averaged treatment effect estimators in different scenarios where they are misspecified for the true underlying treatment effect structure. We show that the immediate treatment effect estimator is relatively robust to bias when estimating a true underlying calendar time-averaged treatment effect estimand. However, when there is a true underlying calendar (exposure) time-varying treatment effect, misspecifying an analysis with an exposure (calendar) time-averaged treatment effect estimator can yield severely misleading estimates and even converge to a value of the opposite sign of the true calendar (exposure) time-averaged treatment effect estimand. In this article, we highlight the two different time scales on which treatment effects can vary in SW-CRTs and clarify potential vulnerabilities that may arise when considering different types of time-varying treatment effects in a SW design. Accordingly, we emphasize the need for researchers to carefully consider whether the treatment effect may vary as a function of exposure time and/or calendar time in the analysis of SW-CRTs.