ReCAP: Recursive Cross Attention Network for Pseudo-Label Generation in Robotic Surgical Skill Assessment

TL;DR

This paper introduces ReCAP, a recursive cross attention network that generates pseudo-labels for surgical skill assessment by tracking OSATS scores throughout a session, improving the interpretability and accuracy of evaluations.

Contribution

We propose a weakly-supervised recurrent transformer model that predicts OSATS scores from kinematic data, enabling detailed performance tracking and improved GRS prediction in surgical skill assessment.

Findings

Outperforms state-of-the-art in GRS prediction using kinematic data (SCC 0.83-0.88)

Matches state-of-the-art performance with video data

Achieves high accuracy in predicting specific OSATS scores (SCC 0.56-0.95)

Abstract

In surgical skill assessment, the Objective Structured Assessments of Technical Skills (OSATS) and Global Rating Scale (GRS) are well-established tools for evaluating surgeons during training. These metrics, along with performance feedback, help surgeons improve and reach practice standards. Recent research on the open-source JIGSAWS dataset, which includes both GRS and OSATS labels, has focused on regressing GRS scores from kinematic data, video, or their combination. However, we argue that regressing GRS alone is limiting, as it aggregates OSATS scores and overlooks clinically meaningful variations during a surgical trial. To address this, we developed a weakly-supervised recurrent transformer model that tracks a surgeon's performance throughout a session by mapping hidden states to six OSATS, derived from kinematic data. These OSATS scores are averaged to predict GRS, allowing us to compare our model's performance against state-of-the-art (SOTA) methods. We report Spearman's Correlation Coefficients (SCC) demonstrating that our model outperforms SOTA using kinematic data (SCC 0.83-0.88), and matches performance with video-based models. Our model also surpasses SOTA in most tasks for average OSATS predictions (SCC 0.46-0.70) and specific OSATS (SCC 0.56-0.95). The generation of pseudo-labels at the segment level translates quantitative predictions into qualitative feedback, vital for automated surgical skill assessment pipelines. A senior surgeon validated our model's outputs, agreeing with 77\% of the weakly-supervised predictions \(p=0.006\).