Scoring and Assessment in Medical VR Training Simulators with Dynamic Time Series Classification

TL;DR

This paper develops and evaluates time-series classification methods for scoring and assessing medical VR training simulators, enabling real-time feedback and skill improvement for trainees.

Contribution

It introduces the DTW-MP method and compares various classification algorithms for VR motion data, demonstrating improved accuracy in trainee skill assessment.

Findings

Deep Learning ResNet achieved 85% accuracy.

Classification accuracy varied from 28.5% to 85%.

Expert data can guide and improve trainee performance.

Abstract

This research proposes and evaluates scoring and assessment methods for Virtual Reality (VR) training simulators. VR simulators capture detailed n-dimensional human motion data which is useful for performance analysis. Custom made medical haptic VR training simulators were developed and used to record data from 271 trainees of multiple clinical experience levels. DTW Multivariate Prototyping (DTW-MP) is proposed. VR data was classified as Novice, Intermediate or Expert. Accuracy of algorithms applied for time-series classification were: dynamic time warping 1-nearest neighbor (DTW-1NN) 60%, nearest centroid SoftDTW classification 77.5%, Deep Learning: ResNet 85%, FCN 75%, CNN 72.5% and MCDCNN 28.5%. Expert VR data recordings can be used for guidance of novices. Assessment feedback can help trainees to improve skills and consistency. Motion analysis can identify different techniques used by individuals. Mistakes can be detected dynamically in real-time, raising alarms to prevent injuries.