MedNgage: A Dataset for Understanding Engagement in Patient-Nurse Conversations

TL;DR

This paper introduces MedNgage, a new dataset of patient-nurse conversations about cancer symptom management, with annotations on engagement, and demonstrates models predicting engagement levels to improve patient care understanding.

Contribution

The paper presents MedNgage, a novel annotated dataset for patient engagement, and shows how transformer models can predict engagement in healthcare conversations.

Findings

Positive correlation between engagement and symptom management outcomes

Transformer models can reliably predict engagement classes

Analysis reveals challenges faced by current models

Abstract

Patients who effectively manage their symptoms often demonstrate higher levels of engagement in conversations and interventions with healthcare practitioners. This engagement is multifaceted, encompassing cognitive and socio-affective dimensions. Consequently, it is crucial for AI systems to understand the engagement in natural conversations between patients and practitioners to better contribute toward patient care. In this paper, we present a novel dataset (MedNgage), which consists of patient-nurse conversations about cancer symptom management. We manually annotate the dataset with a novel framework of categories of patient engagement from two different angles, namely: i) socio-affective (3.1K spans), and ii) cognitive use of language (1.8K spans). Through statistical analysis of the data that is annotated using our framework, we show a positive correlation between patient symptom management outcomes and their engagement in conversations. Additionally, we demonstrate that pre-trained transformer models fine-tuned on our dataset can reliably predict engagement classes in patient-nurse conversations. Lastly, we use LIME (Ribeiro et al., 2016) to analyze the underlying challenges of the tasks that state-of-the-art transformer models encounter. The de-identified data is available for research purposes upon request.