Assessing Health Care Interventions via an Interrupted Time Series Model: Study Power and Design Considerations

TL;DR

This paper introduces the R-MITS model, a new statistical approach for analyzing multi-unit interrupted time series data, enabling better detection of intervention effects and change points in healthcare quality improvement studies.

Contribution

The paper develops the R-MITS model, allowing simultaneous analysis of multiple units and testing for change points, which was not possible with existing ITS methods.

Findings

R-MITS accurately detects change points in simulated data.

The model maintains appropriate error rates in tests.

Application to hospital data demonstrates practical utility.

Abstract

The delivery and assessment of quality health care is complex with many interacting and interdependent components. In terms of research design and statistical analysis, this complexity and interdependency makes it difficult to assess the true impact of interventions designed to improve patient health care outcomes. Interrupted time series (ITS) is a quasi-experimental design developed for inferring the effectiveness of a health policy intervention while accounting for temporal dependence within a single system or unit. Current standardized ITS methods do not simultaneously analyze data for several units, nor are there methods to test for the existence of a change point and to assess statistical power for study planning purposes in this context. To address this limitation we propose the `Robust Multiple ITS' (R-MITS) model, appropriate for multi-unit ITS data, that allows for inference regarding the estimation of a global change point across units in the presence of a potentially lagged (or anticipatory) treatment effect. Under the R-MITS model, one can formally test for the existence of a change point and estimate the time delay between the formal intervention implementation and the over-all-unit intervention effect. We conducted empirical simulation studies to assess the type one error rate of the testing procedure, power for detecting specified change-point alternatives, and accuracy of the proposed estimating methodology. R-MITS is illustrated by analyzing patient satisfaction data from a hospital that implemented and evaluated a new care delivery model in multiple units.