A shared latent space matrix factorisation method for recommending new trial evidence for systematic review updates

TL;DR

This paper introduces a shared latent space matrix factorisation method to improve the identification of relevant clinical trial registrations for systematic review updates, reducing manual screening effort.

Contribution

The study presents a novel matrix factorisation approach that outperforms document similarity methods in ranking relevant trial registrations for systematic review updates.

Findings

Matrix factorisation outperforms document similarity in median rank and recall@100.

Median rank improved from 138 to 59 with the new method.

Recall@100 increased from 42.8% to 60.9% using matrix factorisation.

Abstract

Clinical trial registries can be used to monitor the production of trial evidence and signal when systematic reviews become out of date. However, this use has been limited to date due to the extensive manual review required to search for and screen relevant trial registrations. Our aim was to evaluate a new method that could partially automate the identification of trial registrations that may be relevant for systematic review updates. We identified 179 systematic reviews of drug interventions for type 2 diabetes, which included 537 clinical trials that had registrations in ClinicalTrials.gov. We tested a matrix factorisation approach that uses a shared latent space to learn how to rank relevant trial registrations for each systematic review, comparing the performance to document similarity to rank relevant trial registrations. The two approaches were tested on a holdout set of the newest trials from the set of type 2 diabetes systematic reviews and an unseen set of 141 clinical trial registrations from 17 updated systematic reviews published in the Cochrane Database of Systematic Reviews. The matrix factorisation approach outperformed the document similarity approach with a median rank of 59 and recall@100 of 60.9%, compared to a median rank of 138 and recall@100 of 42.8% in the document similarity baseline. In the second set of systematic reviews and their updates, the highest performing approach used document similarity and gave a median rank of 67 (recall@100 of 62.9%). The proposed method was useful for ranking trial registrations to reduce the manual workload associated with finding relevant trials for systematic review updates. The results suggest that the approach could be used as part of a semi-automated pipeline for monitoring potentially new evidence for inclusion in a review update.