Determining the extent and frequency of on-site monitoring: a bayesian risk-based approach

TL;DR

This paper introduces a Bayesian-based risk boundary method to optimize on-site monitoring in clinical trials, improving efficiency and transparency.

Contribution

A novel Bayesian risk-based approach for determining on-site monitoring extent and frequency, offering transparency and flexibility.

Findings

The proposed Bayesian risk boundaries outperform existing methods in identifying optimal monitoring strategies.

The method is applicable to various clinical trial endpoints and can be used for both trial- and site-level monitoring.

Key factors influencing the performance of the risk boundaries were identified through simulations and real data.

Abstract

On-site monitoring is a crucial component of quality control in clinical trials. However, many cast doubt on its cost-effectiveness due to various issues, such as a lack of monitoring focus that could assist in prioritizing limited resources during a site visit. Consequently, an increasing number of trial sponsors are implementing a hybrid monitoring strategy that combines on-site monitoring with centralised monitoring. One of the primary objectives of centralised monitoring, as stated in the clinical trial guidelines, is to guide and adjust the extent and frequency of on-site monitoring. Quality tolerance limits (QTLs) introduced in ICH E6(R2) and thresholds proposed by TransCelerate Biopharma are two existing approaches for achieving this objective at the trial- and site-levels, respectively. The funnel plot, as another threshold-based site-level method, overcomes the limitation of TransCelerate’s method by adjusting thresholds flexibly based on site sizes. Nonetheless, both methods do not transparently explain the reason for choosing the thresholds that they used or whether their choices are optimal in any certain sense. Additionally, related Bayesian monitoring methods are also lacking.

We propose a simple, transparent, and user-friendly Bayesian-based risk boundary for determining the extent and frequency of on-site monitoring both at the trial- and site-levels. We developed a four-step approach, including: 1) establishing risk levels for key risk indicators (KRIs) along with their corresponding monitoring actions and estimates; 2) calculating the optimal risk boundaries; 3) comparing the outcomes of KRIs against the optimal risk boundaries; and 4) providing recommendations based on the comparison results. Our method can be used to identify the optimal risk boundaries within an established risk level range and is applicable to continuous, discrete, and time-to-event endpoints.

We evaluate the performance of the proposed risk boundaries via simulations that mimic various realistic clinical trial scenarios. The performance of the proposed risk boundaries is compared against the funnel plot using real clinical trial data. The results demonstrate the applicability and flexibility of the proposed method for clinical trial monitoring. Moreover, we identify key factors that affect the optimality and performance of the proposed risk boundaries, respectively.

Given the aforementioned advantages of the proposed risk boundaries, we expect that they will benefit the clinical trial community at large, in particular in the realm of risk-based monitoring.

The online version contains supplementary material available at 10.1186/s12874-024-02261-y.

Figures