Rate of Orientation Change as a New Metric for Robot-Assisted and Open Surgical Skill Evaluation

TL;DR

This paper introduces a novel metric based on the rate of orientation change of surgical instruments to objectively evaluate surgical skills, demonstrating its effectiveness alongside classical metrics across different datasets.

Contribution

The study develops and validates a new orientation-based metric for surgical skill assessment, emphasizing the importance of movement segmentation and providing a more comprehensive evaluation tool.

Findings

Rate of orientation change significantly differentiates skill levels.

Segmentation improves metric sensitivity and accuracy.

New metric complements classical measures like task time.

Abstract

Surgeons' technical skill directly impacts patient outcomes. To date, the angular motion of the instruments has been largely overlooked in objective skill evaluation. To fill this gap, we have developed metrics for surgical skill evaluation that are based on the orientation of surgical instruments. We tested our new metrics on two datasets with different conditions: (1) a dataset of experienced robotic surgeons and nonmedical users performing needle-driving on a dry lab model, and (2) a small dataset of suturing movements performed by surgeons training on a porcine model. We evaluated the performance of our new metrics (angular displacement and the rate of orientation change) alongside the performances of classical metrics (task time and path length). We calculated each metric on different segments of the movement. Our results highlighted the importance of segmentation rather than calculating the metrics on the entire movement. Our new metric, the rate of orientation change, showed statistically significant differences between experienced surgeons and nonmedical users / novice surgeons, which were consistent with the classical task time metric. The rate of orientation change captures technical aspects that are taught during surgeons' training, and together with classical metrics can lead to a more comprehensive discrimination of skills.