Design and analysis of behavioral intervention studies: A Bayesian approach

TL;DR

This paper explains how Bayesian methods can improve the design and analysis of behavioral intervention studies compared to traditional statistical approaches.

Contribution

The paper introduces a Bayesian framework for a priori sample size determination in behavioral intervention studies.

Findings

Bayesian methods like Bayes factors and posterior model probabilities offer advantages over null hypothesis significance testing.

A criterion and procedure for Bayesian a priori sample size determination are proposed and illustrated.

The methodology is demonstrated using a real-world cluster randomized trial dataset in R.

Abstract

To study the effect of a behavioral intervention, it should be compared to a control or an existing treatment in an intervention study. There exist many guidelines in the literature about the design and analysis of intervention studies, including recommendations for a priori sample size determination. The vast majority of these guidelines are based on the framework of null hypothesis significance testing, where a p-value is compared to a user-selected type I error rate to determine whether an effect is significant or not. This approach has received severe criticism over the past decades as it has resulted in publication bias, sloppy science, and fraud. The Bayesian approach to hypothesis testing has been developed to overcome some of these drawbacks. The Bayes factor quantifies the relative support in the data for one hypothesis over another hypothesis. The hypotheses do not necessarily have to include a null hypothesis and can be formulated based on observations, findings in the literature, or an expert’s opinion. Posterior Model Probabilities, which are a function of the Bayes Factor, can be used to compare a set of hypotheses to one another and select the one most supported by the data. In this paper, we summarize the shortcomings of null hypothesis significance testing, introduce the Bayes factor and Posterior Model Probabilities, explain how they are calculated, and how they are interpreted. We also focus on a priori sample size determination in the Bayesian hypothesis testing framework. We introduce a criterion for sample size determination and a procedure to find the required sample size. We illustrate our methodology using a cluster randomized trial on the effectiveness of an online training in improving primary care doctors’ competency in brief tobacco interventions. All analyses are done in R, and we provide the dataset and R syntax for straightforward replication.

Figures