An Analysis of a BERT Deep Learning Strategy on a Technology Assisted Review Task

TL;DR

This paper presents a deep learning approach using BERT-based embeddings for document classification and retrieval in clinical evidence review, demonstrating improved initial ranking performance over traditional models on CLEF eHealth datasets.

Contribution

The study introduces a novel DL strategy combining BERT and SBERT embeddings for clinical document retrieval, outperforming traditional models in initial ranking effectiveness.

Findings

DL strategy achieves high retrieval effectiveness

Outperforms BM25 plus RM3 model in initial ranking

Validated on CLEF eHealth collections

Abstract

Document screening is a central task within Evidenced Based Medicine, which is a clinical discipline that supplements scientific proof to back medical decisions. Given the recent advances in DL (Deep Learning) methods applied to Information Retrieval tasks, I propose a DL document classification approach with BERT or PubMedBERT embeddings and a DL similarity search path using SBERT embeddings to reduce physicians' tasks of screening and classifying immense amounts of documents to answer clinical queries. I test and evaluate the retrieval effectiveness of my DL strategy on the 2017 and 2018 CLEF eHealth collections. I find that the proposed DL strategy works, I compare it to the recently successful BM25 plus RM3 model, and conclude that the suggested method accomplishes advanced retrieval performance in the initial ranking of the articles with the aforementioned datasets, for the CLEF eHealth Technologically Assisted Reviews in Empirical Medicine Task.