Anytime-valid inference in N-of-1 trials

TL;DR

This paper introduces a framework for valid interim analysis in N-of-1 trials, allowing continuous inference during the trial without inflating error rates, thus improving participant engagement and trial efficiency.

Contribution

It develops a potential outcomes framework with confidence sequences for anytime-valid inference, enabling early and valid results in N-of-1 trials.

Findings

Valid confidence sequences are achievable over time.

The approach maintains statistical validity during interim peeks.

Empirical results support the method's effectiveness.

Abstract

App-based N-of-1 trials offer a scalable experimental design for assessing the effects of health interventions at an individual level. Their practical success depends on the strong motivation of participants, which, in turn, translates into high adherence and reduced loss to follow-up. One way to maintain participant engagement is by sharing their interim results. Continuously testing hypotheses during a trial, known as "peeking", can also lead to shorter, lower-risk trials by detecting strong effects early. Nevertheless, traditionally, results are only presented upon the trial's conclusion. In this work, we introduce a potential outcomes framework that permits interim peeking of the results and enables statistically valid inferences to be drawn at any point during N-of-1 trials. Our work builds on the growing literature on valid confidence sequences, which enables anytime-valid inference with uniform type-1 error guarantees over time. We propose several causal estimands for treatment effects applicable in an N-of-1 trial and demonstrate, through empirical evaluation, that the proposed approach results in valid confidence sequences over time. We anticipate that incorporating anytime-valid inference into clinical trials can significantly enhance trial participation and empower participants.