FAMULUS: Interactive Annotation and Feedback Generation for Teaching Diagnostic Reasoning

TL;DR

FAMULUS is an interactive system that provides automatic feedback and supports data annotation in virtual patient simulations to enhance diagnostic reasoning training for students and instructors.

Contribution

It introduces a novel interactive approach combining NLP-based automatic feedback and annotation support for diagnostic training systems.

Findings

System improves diagnostic reasoning skills in students.

Recurrent use accelerates data annotation and system improvement.

User studies demonstrate effectiveness in medical and teacher education.

Abstract

Our proposed system FAMULUS helps students learn to diagnose based on automatic feedback in virtual patient simulations, and it supports instructors in labeling training data. Diagnosing is an exceptionally difficult skill to obtain but vital for many different professions (e.g., medical doctors, teachers). Previous case simulation systems are limited to multiple-choice questions and thus cannot give constructive individualized feedback on a student's diagnostic reasoning process. Given initially only limited data, we leverage a (replaceable) NLP model to both support experts in their further data annotation with automatic suggestions, and we provide automatic feedback for students. We argue that because the central model consistently improves, our interactive approach encourages both students and instructors to recurrently use the tool, and thus accelerate the speed of data creation and annotation. We show results from two user studies on diagnostic reasoning in medicine and teacher education and outline how our system can be extended to further use cases.