LLM-Powered Virtual Patient Agents for Interactive Clinical Skills Training with Automated Feedback

TL;DR

This paper introduces an advanced LLM-based virtual patient system with action capabilities and real-time feedback, enhancing medical training efficiency and realism for OSCE preparation.

Contribution

It presents a novel framework that adds action spaces to LLM virtual patients and integrates virtual tutors for instant personalized feedback, improving realism and educational value.

Findings

System demonstrates low latency and high accuracy in real-time interactions.

Preliminary expert evaluations confirm naturalness and coherence of virtual patients.

Virtual tutors provide useful, appropriate feedback enhancing learning outcomes.

Abstract

Objective Structured Clinical Examinations (OSCEs) are essential for medical training, but they require significant resources, including professional actors and expert medical feedback. Although Large Language Models (LLMs) have introduced text-based virtual patients for communication practice, these simulations often lack the capability for richer, non-textual interactions. This paper presents a novel framework that significantly enhances LLM-based simulated patients by equipping them with action spaces, thereby enabling more realistic and dynamic patient behaviors that extend beyond text. Furthermore, our system incorporates virtual tutors that provide students with instant, personalized feedback on their performance at any time during these simulated encounters. We have conducted a rigorous evaluation of the framework's real-time performance, including system latency and component accuracy. Preliminary evaluations with medical experts assessed the naturalness and coherence of the simulated patients, as well as the usefulness and appropriateness of the virtual tutor's assessments. This innovative system provides medical students with a low-cost, accessible platform for personalized OSCE preparation at home.