How to design a pre-specified statistical analysis approach to limit p-hacking in clinical trials: the Pre-SPEC framework

TL;DR

This paper introduces the Pre-SPEC framework, a five-point method for designing pre-specified statistical analysis plans in clinical trials to prevent p-hacking and ensure unbiased results.

Contribution

It provides a detailed framework to improve pre-specification of analysis plans, reducing bias and enhancing transparency in clinical trial reporting.

Findings

Framework guides detailed pre-specification of analysis methods

Helps prevent p-hacking in clinical trials

Aligns with existing trial protocol guidelines

Abstract

Results from clinical trials can be susceptible to bias if investigators choose their analysis approach after seeing trial data, as this can allow them to perform multiple analyses and then choose the method that provides the most favourable result (commonly referred to as 'p-hacking'). Pre-specification of the planned analysis approach is essential to help reduce such bias, as it ensures analytical methods are chosen in advance of seeing the trial data. However, pre-specification is only effective if done in a way that does not allow p-hacking. For example, investigators may pre-specify a certain statistical method such as multiple imputation, but give little detail on how it will be implemented. Because there are many different ways to perform multiple imputation, this approach to pre-specification is ineffective, as it still allows investigators to analyse the data in different ways before deciding on a final approach. In this article we describe a five-point framework (the Pre-SPEC framework) for designing a pre-specified analysis approach that does not allow p-hacking. This framework is intended to be used in conjunction with the SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) statement and other similar guidelines to help investigators design the statistical analysis strategy for the trial's primary outcome in the trial protocol.