Medical Literature Mining and Retrieval in a Conversational Setting

TL;DR

This paper presents a conversational system that combines neural and traditional retrieval methods to efficiently extract and present coronavirus-related medical literature in a user-friendly, multi-turn dialogue format.

Contribution

It introduces a novel system integrating DialoGPT for conversation and an ensemble retrieval approach combining BM-25 and neural embeddings for effective literature search.

Findings

Neural embedding-based retrieval outperforms BM-25 in accuracy.

The system effectively answers COVID-19 related queries.

Experimental results validate the ensemble retrieval approach.

Abstract

The Covid-19 pandemic has caused a spur in the medical research literature. With new research advances in understanding the virus, there is a need for robust text mining tools which can process, extract and present answers from the literature in a concise and consumable way. With a DialoGPT based multi-turn conversation generation module, and BM-25 \& neural embeddings based ensemble information retrieval module, in this paper we present a conversational system, which can retrieve and answer coronavirus-related queries from the rich medical literature, and present it in a conversational setting with the user. We further perform experiments to compare neural embedding-based document retrieval and the traditional BM25 retrieval algorithm and report the results.