From Protocol to Screening: A Hybrid Learning Approach for Technology-Assisted Systematic Literature Reviews

TL;DR

This paper introduces a comprehensive hybrid learning pipeline for technology-assisted systematic literature reviews in medicine, reducing manual effort and improving retrieval accuracy through innovative ranking techniques.

Contribution

It presents a novel full pipeline from protocol to screening that eliminates the need for Boolean queries and combines multiple learning-to-rank components.

Findings

Achieves state-of-the-art results on the CLEF 2019 dataset.

Demonstrates effective integration of relevance feedback in TAR.

Provides an updated, publicly available dataset for future research.

Abstract

In the medical domain, a Systematic Literature Review (SLR) attempts to collect all empirical evidence, that fit pre-specified eligibility criteria, in order to answer a specific research question. The process of preparing an SLR consists of multiple tasks that are labor-intensive and time-consuming, involving large monetary costs. Technology-assisted review (TAR) methods automate the different processes of creating an SLR and they are particularly focused on reducing the burden of screening for reviewers. We present a novel method for TAR that implements a full pipeline from the research protocol to the screening of the relevant papers. Our pipeline overcomes the need of a Boolean query constructed by specialists and consists of three different components: the primary retrieval engine, the inter-review ranker and the intra-review ranker, combining learning-to-rank techniques with a relevance feedback method. In addition, we contribute an updated version of the Task 2 of the CLEF 2019 eHealth Lab dataset, which we make publicly available. Empirical results on this dataset show that our approach can achieve state-of-the-art results.