An automated approach to extracting positive and negative clinical research results

TL;DR

This paper presents an automated method for extracting and classifying positive and negative results from clinical trial reports using a PICOE framework, aiding systematic understanding of clinical evidence.

Contribution

The study introduces a pipeline that accurately extracts statistical effects from RCT reports and distinguishes negative results, revealing insights into reporting biases in clinical literature.

Findings

Negative results constitute nearly 40% of Covid-19 intervention-outcome pairs.

High accuracy in extracting ICO and effect descriptive words.

The tool helps systematically analyze clinical evidence and reporting biases.

Abstract

Failure is common in clinical trials since the successful failures presented in negative results always indicate the ways that should not be taken. In this paper, we proposed an automated approach to extracting positive and negative clinical research results by introducing a PICOE (Population, Intervention, Comparation, Outcome, and Effect) framework to represent randomized controlled trials (RCT) reports, where E indicates the effect between a specific I and O. We developed a pipeline to extract and assign the corresponding statistical effect to a specific I-O pair from natural language RCT reports. The extraction models achieved a high degree of accuracy for ICO and E descriptive words extraction through two rounds of training. By defining a threshold of p-value, we find in all Covid-19 related intervention-outcomes pairs with statistical tests, negative results account for nearly 40%. We believe that this observation is noteworthy since they are extracted from the published literature, in which there is an inherent risk of reporting bias, preferring to report positive results rather than negative results. We provided a tool to systematically understand the current level of clinical evidence by distinguishing negative results from the positive results.