Dense Retrieval with Continuous Explicit Feedback for Systematic Review Screening Prioritisation

TL;DR

This paper introduces a neural dense representation method that uses continuous relevance feedback to improve screening prioritisation in systematic reviews, avoiding costly model fine-tuning and inference.

Contribution

It presents a novel approach leveraging relevance feedback to update dense query representations for efficient document ranking without fine-tuning neural models.

Findings

The dense feedback method outperforms direct neural model use in efficiency.

It shows promising effectiveness compared to previous screening prioritisation methods.

The approach is validated on CLEF TAR datasets.

Abstract

The goal of screening prioritisation in systematic reviews is to identify relevant documents with high recall and rank them in early positions for review. This saves reviewing effort if paired with a stopping criterion, and speeds up review completion if performed alongside downstream tasks. Recent studies have shown that neural models have good potential on this task, but their time-consuming fine-tuning and inference discourage their widespread use for screening prioritisation. In this paper, we propose an alternative approach that still relies on neural models, but leverages dense representations and relevance feedback to enhance screening prioritisation, without the need for costly model fine-tuning and inference. This method exploits continuous relevance feedback from reviewers during document screening to efficiently update the dense query representation, which is then applied to rank the remaining documents to be screened. We evaluate this approach across the CLEF TAR datasets for this task. Results suggest that the investigated dense query-driven approach is more efficient than directly using neural models and shows promising effectiveness compared to previous methods developed on the considered datasets. Our code is available at https://github.com/ielab/dense-screening-feedback.