SurgX: Neuron-Concept Association for Explainable Surgical Phase Recognition

TL;DR

SurgX introduces a neuron-concept association framework that improves interpretability of surgical phase recognition models, aiding understanding, trust, and debugging of deep learning in surgical workflow analysis.

Contribution

The paper presents a novel concept-based explanation method that links neurons to relevant surgical concepts, enhancing interpretability of phase recognition models.

Findings

Validated on two models with extensive experiments

Improved interpretability of surgical phase recognition

Potential to increase trust and facilitate debugging

Abstract

Surgical phase recognition plays a crucial role in surgical workflow analysis, enabling various applications such as surgical monitoring, skill assessment, and workflow optimization. Despite significant advancements in deep learning-based surgical phase recognition, these models remain inherently opaque, making it difficult to understand how they make decisions. This lack of interpretability hinders trust and makes it challenging to debug the model. To address this challenge, we propose SurgX, a novel concept-based explanation framework that enhances the interpretability of surgical phase recognition models by associating neurons with relevant concepts. In this paper, we introduce the process of selecting representative example sequences for neurons, constructing a concept set tailored to the surgical video dataset, associating neurons with concepts and identifying neurons crucial for predictions. Through extensive experiments on two surgical phase recognition models, we validate our method and analyze the explanation for prediction. This highlights the potential of our method in explaining surgical phase recognition. The code is available at https://github.com/ailab-kyunghee/SurgX