Learning to Ask Like a Physician

TL;DR

This paper introduces DiSCQ, a new dataset of over 2,000 clinically relevant questions generated by medical experts from discharge summaries, aiming to improve realistic clinical question answering and question generation models.

Contribution

The creation of the DiSCQ dataset with expert-generated questions and triggers, and baseline models for trigger detection and question generation in clinical settings.

Findings

Baseline models generate high-quality questions in over 62% of cases.

The dataset captures realistic physician information needs.

Code and dataset are publicly released for further research.

Abstract

Existing question answering (QA) datasets derived from electronic health records (EHR) are artificially generated and consequently fail to capture realistic physician information needs. We present Discharge Summary Clinical Questions (DiSCQ), a newly curated question dataset composed of 2,000+ questions paired with the snippets of text (triggers) that prompted each question. The questions are generated by medical experts from 100+ MIMIC-III discharge summaries. We analyze this dataset to characterize the types of information sought by medical experts. We also train baseline models for trigger detection and question generation (QG), paired with unsupervised answer retrieval over EHRs. Our baseline model is able to generate high quality questions in over 62% of cases when prompted with human selected triggers. We release this dataset (and all code to reproduce baseline model results) to facilitate further research into realistic clinical QA and QG: https://github.com/elehman16/discq.