Semantic Similarity To Improve Question Understanding in a Virtual Patient

TL;DR

This paper presents a virtual patient system that uses semantic similarity with distributed word representations to improve question understanding, achieving high accuracy in clinical case simulations for medical students.

Contribution

The study introduces a hybrid dialogue system combining rule-based and semantic similarity methods, enhancing question understanding in virtual medical consultations.

Findings

Semantic similarity improves question matching accuracy.

Hybrid system outperforms rule-based system by 2.59% in F1-score.

Error reduction of 9.70% with combined approach.

Abstract

In medicine, a communicating virtual patient or doctor allows students to train in medical diagnosis and develop skills to conduct a medical consultation. In this paper, we describe a conversational virtual standardized patient system to allow medical students to simulate a diagnosis strategy of an abdominal surgical emergency. We exploited the semantic properties captured by distributed word representations to search for similar questions in the virtual patient dialogue system. We created two dialogue systems that were evaluated on datasets collected during tests with students. The first system based on hand-crafted rules obtains $92.29\%$ as $F1$-score on the studied clinical case while the second system that combines rules and semantic similarity achieves $94.88\%$. It represents an error reduction of $9.70\%$ as compared to the rules-only-based system.