Identifying novel indicators of non-technical skills derived from operative video annotation

TL;DR

This study shows that non-technical surgical skills can be measured from video data, offering a scalable alternative to expert observation.

Contribution

Gesture-based metrics from video annotations predict cognitive non-technical skills in surgery.

Findings

Five video-derived indicators explained 39.6% of the variance in expert NTS ratings.

Dexterity index and specific temporal features were key predictors of cognitive skills.

Decision-making and situation awareness were highly correlated in expert ratings.

Abstract

Cognitive non-technical skills (NTS), including situation awareness and decision-making, are critical determinants of surgical outcomes. Current NTS assessments depend on expert human observation, which is resource-intensive and difficult to scale. To address this, we investigated whether surgical gestures, derived from annotated video of the surgical field, could serve as objective indicators of cognitive NTS.

A data set of 40 open-source laparoscopic appendicectomy videos was annotated for temporal (for example surgical gestures) and spatial (for example coordinates of actions) indicators of surgical NTS. Using the Non-Technical Skills for Surgeons (NOTSS) tool, 12 expert observers independently assessed decision-making (1–4) and situation awareness (1–4). Multivariable linear regression analysed video-derived indicators predictive of NTS.

Across all videos, a total of 10 385 events were annotated, generating 87 374 data points. The mean cognitive NOTSS rating was 5.6 (s.d. 0.8) out of 8, with decision-making and situation awareness highly correlated (r = 0.8, P  <  0.001). The final multivariable model explained 39.6% of the variance in expert cognitive NOTSS ratings, identifying five predictors: dexterity index, events during the final operative phase, coagulating events, dropping actions, and actions targeting the small bowel.

This study provides evidence that non-technical skills can be inferred from silent video of the surgical field alone. These findings lay the foundations for scalable, automated tools to evaluate surgeons’ cognitive processes offering new avenues to improve surgical training, performance and outcomes.

Developing data-driven insights into cognitive non-technical skills would offer an alternative to resource-intensive expert observation. Using annotated laparoscopic appendectomy videos, gesture-based metrics accounted for a substantial proportion of variance in expert ratings, with specific temporal features emerging as key predictors. These findings support the feasibility of scalable, automated assessment tools to inform training and enhance operative performance.

Figures