ZEAL: Surgical Skill Assessment with Zero-shot Tool Inference Using Unified Foundation Model

TL;DR

ZEAL introduces a zero-shot, foundation model-based method for surgical skill assessment that leverages segmentation masks and temporal modeling to objectively evaluate surgical proficiency without extensive training data.

Contribution

This work presents ZEAL, a novel zero-shot approach combining foundation models, segmentation, and LSTM networks for surgical skill evaluation, reducing reliance on labeled datasets.

Findings

ZEAL outperforms traditional methods on open datasets.

It effectively captures instrument and environment features.

The approach enables objective, real-time surgical skill assessment.

Abstract

Surgical skill assessment is paramount for ensuring patient safety and enhancing surgical outcomes. This study addresses the need for efficient and objective evaluation methods by introducing ZEAL (surgical skill assessment with Zero-shot surgical tool segmentation with a unifiEd foundAtion modeL). ZEAL uses segmentation masks of surgical instruments obtained through a unified foundation model for proficiency assessment. Through zero-shot inference with text prompts, ZEAL predicts segmentation masks, capturing essential features of both instruments and surroundings. Utilizing sparse convolutional neural networks and segmentation masks, ZEAL extracts feature vectors for foreground (instruments) and background. Long Short-Term Memory (LSTM) networks encode temporal dynamics, modeling sequential data and dependencies in surgical videos. Combining LSTM-encoded vectors, ZEAL produces a surgical skill score, offering an objective measure of proficiency. Comparative analysis with conventional methods using open datasets demonstrates ZEAL's superiority, affirming its potential in advancing surgical training and evaluation. This innovative approach to surgical skill assessment addresses challenges in traditional supervised learning techniques, paving the way for enhanced surgical care quality and patient outcomes.