Application of the Composite Quality Score (CQS-2B) versus Cochrane’s Risk of Bias tool (Version 2) in systematic reviews of clinical trials – an exploratory study

TL;DR

This study compares two tools for evaluating clinical trial quality and finds that one may lead to more conservative conclusions.

Contribution

The study introduces a hypothesis that the CQS-2B tool may yield more conservative conclusions than RoB 2 in systematic reviews.

Findings

Pooled effect estimates using RoB 2 and CQS-2B showed no statistically significant difference.

Clinical conclusions aligned with RoB 2 but not with CQS-2B due to perceived overestimation.

A hypothesis was generated that CQS-2B may provide more conservative conclusions.

Abstract

To explore whether systematic review conclusions generated from Cochrane’s second version of its Risk of Bias tool (RoB 2) for trial appraisal differ when the Composite Quality Score, Version 2.B (CQS-2B) is used instead and to develop a testable hypothesis based on these findings.

PubMed was searched for one single systematic review. From the review’s accepted trials, data concerning effect estimates and overall bias risk according to the RoB 2 tool were extracted. All trial reports were appraised again using the CQS-2B. Datasets were stratified according to overall bias risk (RoB 2) or corroboration (C-) level (CQS-2B). The effect estimates from trials with ‘low bias risk’ (RoB 2) and with highest C-level (CQS-2B) were pooled separately. These pooled effect estimates were statistically and all clinical conclusions qualitatively compared.

The pooled effect estimates for trials with ‘low bias risk’ (RoB 2) were −0.07, 95% CI: −0.10 to −0.04 (I2 = 0.0%) and for the highest C-levels (CQS-2B) 0.08, 95% CI: −0.12 to −0.04 (I2 = 57.0%). The difference was statistically not significant (p = 0.70). Contrary to the RoB 2 tool, no clinical conclusions in line with the CQS-2B were made, because the effect estimates were judged to be erroneously overestimated, due to high risk of bias.

A testable hypothesis was generated suggesting that trial appraisal using the CQS-2B may provide more conservative conclusions based on similar data than with the RoB 2 tool.

Figures