Know your sensORs -- A Modality Study For Surgical Action Classification

TL;DR

This paper investigates how different sensor modalities affect surgical action recognition in operating rooms, demonstrating that modality choice impacts performance and proposing fusion methods to enhance classification accuracy.

Contribution

It provides a systematic analysis of sensor modalities for surgical action recognition and introduces fusion approaches that improve classification performance.

Findings

Sensor modality choice influences recognition accuracy.

Fusion methods improve classification performance.

Analysis conducted on multi-view RGB-D videos of laparoscopic surgeries.

Abstract

The surgical operating room (OR) presents many opportunities for automation and optimization. Videos from various sources in the OR are becoming increasingly available. The medical community seeks to leverage this wealth of data to develop automated methods to advance interventional care, lower costs, and improve overall patient outcomes. Existing datasets from OR room cameras are thus far limited in size or modalities acquired, leaving it unclear which sensor modalities are best suited for tasks such as recognizing surgical action from videos. This study demonstrates that surgical action recognition performance can vary depending on the image modalities used. We perform a methodical analysis on several commonly available sensor modalities, presenting two fusion approaches that improve classification performance. The analyses are carried out on a set of multi-view RGB-D video recordings of 18 laparoscopic procedures.