Distinguishing Transformative from Incremental Clinical Evidence: A Classifier of Clinical Research using Textual features from Abstracts and Citing Sentences

TL;DR

This study develops a machine learning classifier to distinguish transformative clinical research from incremental studies by analyzing textual features from abstracts and citing sentences, aiding clinicians and researchers in evaluating research impact.

Contribution

It introduces a novel approach using textual features from abstracts and citing sentences to automatically classify clinical research as transformative or incremental.

Findings

Achieved an average AUC of 0.755 with Random Forest classifier.

Citing sentences contain distinctive language patterns for transformative research.

Provides an efficient tool for identifying impactful clinical evidence.

Abstract

In clinical research and clinical decision-making, it is important to know if a study changes or only supports the current standards of care for specific disease management. We define such a change as transformative and a support as incremental research. It usually requires a huge amount of domain expertise and time for humans to finish such tasks. Faculty Opinions provides us with a well-annotated corpus on whether a research challenges or only confirms established research. In this study, a machine learning approach is proposed to distinguishing transformative from incremental clinical evidence. The texts from both abstract and a 2-year window of citing sentences are collected for a training set of clinical studies recommended and labeled by Faculty Opinions experts. We achieve the best performance with an average AUC of 0.755 (0.705-0.875) using Random Forest as the classifier and citing sentences as the feature. The results showed that transformative research has typical language patterns in citing sentences unlike abstract sentences. We provide an efficient tool for identifying those clinical evidence challenging or only confirming established claims for clinicians and researchers.