Extracting Structured Data from Physician-Patient Conversations By Predicting Noteworthy Utterances

TL;DR

This paper introduces a method to extract structured medical data from lengthy physician-patient conversations by identifying key utterances, thereby improving diagnosis and abnormality recognition for post-visit documentation.

Contribution

It presents a new dataset and a novel approach focusing on predicting noteworthy utterances to enhance information extraction from long medical conversations.

Findings

Filtering for noteworthy utterances improves prediction accuracy

The approach effectively identifies relevant diagnoses and abnormalities

The dataset enables future research in medical conversation analysis

Abstract

Despite diverse efforts to mine various modalities of medical data, the conversations between physicians and patients at the time of care remain an untapped source of insights. In this paper, we leverage this data to extract structured information that might assist physicians with post-visit documentation in electronic health records, potentially lightening the clerical burden. In this exploratory study, we describe a new dataset consisting of conversation transcripts, post-visit summaries, corresponding supporting evidence (in the transcript), and structured labels. We focus on the tasks of recognizing relevant diagnoses and abnormalities in the review of organ systems (RoS). One methodological challenge is that the conversations are long (around 1500 words), making it difficult for modern deep-learning models to use them as input. To address this challenge, we extract noteworthy utterances---parts of the conversation likely to be cited as evidence supporting some summary sentence. We find that by first filtering for (predicted) noteworthy utterances, we can significantly boost predictive performance for recognizing both diagnoses and RoS abnormalities.