The application of ROBINS-I guidance in systematic reviews of non-randomised studies: A descriptive study

TL;DR

This study examines how well systematic reviews follow the ROBINS-I guidance for assessing bias in non-randomized studies, finding that most do not fully adhere to the tool's recommendations.

Contribution

The study provides a descriptive analysis of the application of ROBINS-I guidance in systematic reviews of non-randomized studies.

Findings

Only one of 492 reviews fully met ROBINS-I guidance expectations.

Most reviews reported ROBINS-I results at the study level rather than the outcome level.

Deviation from guidance raises concerns about the validity of reported results.

Abstract

The ROBINS-I tool is a commonly used tool to assess risk of bias in non-randomised studies of interventions (NRSI) included in systematic reviews. The reporting of ROBINS-I results is important for decision-makers using systematic reviews to understand the weaknesses of the evidence. In particular, systematic review authors should apply the tool according to the guidance provided. This study aims to describe how ROBINS-I guidance is currently applied by review authors. In January 2023, we undertook a citation search and screened titles and abstracts of records published in the previous 6 months. We included systematic reviews of non-randomised studies of intervention where ROBINS-I had been used for risk-of-bias assessment. Based on 10 criteria, we summarised the diverse ways in which reviews deviated from or reported the use of ROBINS-I. In total, 492 reviews met our inclusion criteria. Only one review met all the expectations of the ROBINS-I guidance. A small proportion of reviews deviated from the seven standard domains (3%), judgements (13%), or in other ways (1%). Of the 476 (97%) reviews that reported some ROBINS-I results, only 57 (12%) reviews reported ROBINS-I results at the outcome level compared with 203 reviews that reported ROBINS-I results at the study level alone. Most systematic reviews of NRSIs do not fully apply the ROBINS-I guidance. This raises concerns around the validity of the ROBINS-I results reported and the use of the evidence from these reviews in decision-making.

Figures