Objective Surgical Skills Assessment and Tool Localization: Results from the MICCAI 2021 SimSurgSkill Challenge

TL;DR

This paper reports on the MICCAI 2021 SimSurgSkill Challenge, which developed automated methods for surgical skill assessment and tool localization using VR data, aiming to improve surgical training feedback.

Contribution

It introduces a publicly available dataset and summarizes the top-performing machine learning approaches for automated surgical skill evaluation and tool localization.

Findings

Winning methods achieved high localization accuracy.

Automated assessments correlated well with expert evaluations.

The dataset enables future research in surgical data science.

Abstract

Timely and effective feedback within surgical training plays a critical role in developing the skills required to perform safe and efficient surgery. Feedback from expert surgeons, while especially valuable in this regard, is challenging to acquire due to their typically busy schedules, and may be subject to biases. Formal assessment procedures like OSATS and GEARS attempt to provide objective measures of skill, but remain time-consuming. With advances in machine learning there is an opportunity for fast and objective automated feedback on technical skills. The SimSurgSkill 2021 challenge (hosted as a sub-challenge of EndoVis at MICCAI 2021) aimed to promote and foster work in this endeavor. Using virtual reality (VR) surgical tasks, competitors were tasked with localizing instruments and predicting surgical skill. Here we summarize the winning approaches and how they performed. Using this publicly available dataset and results as a springboard, future work may enable more efficient training of surgeons with advances in surgical data science. The dataset can be accessed from https://console.cloud.google.com/storage/browser/isi-simsurgskill-2021.