Concept Graph Neural Networks for Surgical Video Understanding

TL;DR

This paper introduces a novel temporal concept graph network that integrates global knowledge graphs into surgical video analysis, improving recognition and understanding of complex surgical procedures.

Contribution

It presents a new method for incorporating conceptual knowledge into temporal analysis of surgical videos, enhancing interpretability and performance.

Findings

Improved recognition of surgical safety views

Enhanced estimation of Parkland grading scale

Better detection of complex surgical benchmarks

Abstract

We constantly integrate our knowledge and understanding of the world to enhance our interpretation of what we see. This ability is crucial in application domains which entail reasoning about multiple entities and concepts, such as AI-augmented surgery. In this paper, we propose a novel way of integrating conceptual knowledge into temporal analysis tasks via temporal concept graph networks. In the proposed networks, a global knowledge graph is incorporated into the temporal analysis of surgical instances, learning the meaning of concepts and relations as they apply to the data. We demonstrate our results in surgical video data for tasks such as verification of critical view of safety, as well as estimation of Parkland grading scale. The results show that our method improves the recognition and detection of complex benchmarks as well as enables other analytic applications of interest.