Model selection and real-time skill assessment for suturing in robotic surgery

TL;DR

This study develops multimodal deep learning models that integrate kinematic and vision data to provide real-time, objective skill assessment in robotic surgery, demonstrating improved prediction accuracy and generalization across skill levels.

Contribution

It introduces fusion-based multimodal models for real-time surgical skill prediction and evaluates their effectiveness and generalization in a robotic surgery context.

Findings

Fusion models outperform unimodal models in real-time skill prediction.

High-skill training data improves model performance and generalization.

Models correlate well with surgeon gestures and skill levels.

Abstract

Automated feedback systems have the potential to provide objective skill assessment for training and evaluation in robot-assisted surgery. In this study, we examine methods to achieve real-time prediction of surgical skill level in real-time based on Objective Structured Assessment of Technical Skills (OSATS) scores. Using data acquired from the da Vinci Surgical System, we carry out three main analyses, focusing on model design, their real-time performance, and their skill-level-based cross-validation training. For the model design, we evaluate the effectiveness of multimodal deep learning models for predicting surgical skill levels using synchronized kinematic and vision data. Our models include separate unimodal baselines and fusion architectures that integrate features from both modalities and are evaluated using mean Spearman's correlation coefficients, demonstrating that the fusion model consistently outperforms unimodal models for real-time predictions. For the real-time performance, we observe the prediction's trend over time and highlight correlation with the surgeon's gestures. For the skill-level-based cross-validation, we separately trained models on surgeons with different skill levels, which showed that high-skill demonstrations allow for better performance than those trained on low-skilled ones and generalize well to similarly skilled participants. Our findings show that multimodal learning allows more stable fine-grained evaluation of surgical performance and highlights the value of expert-level training data for model generalization.