Single- and Multi-Task Architectures for Surgical Workflow Challenge at M2CAI 2016

TL;DR

This paper compares single-task and multi-task deep learning architectures for surgical phase recognition in laparoscopic videos, demonstrating that LSTM-based temporal modeling outperforms HMM-based methods.

Contribution

It introduces and evaluates multi-task architectures combining phase recognition and tool detection, with a focus on temporal constraint enforcement methods.

Findings

LSTM-based approach outperforms HMM-based approach.

Multi-task architecture improves recognition accuracy.

LSTM effectively enforces temporal constraints.

Abstract

The surgical workflow challenge at M2CAI 2016 consists of identifying 8 surgical phases in cholecystectomy procedures. Here, we propose to use deep architectures that are based on our previous work where we presented several architectures to perform multiple recognition tasks on laparoscopic videos. In this technical report, we present the phase recognition results using two architectures: (1) a single-task architecture designed to perform solely the surgical phase recognition task and (2) a multi-task architecture designed to perform jointly phase recognition and tool presence detection. On top of these architectures we propose to use two different approaches to enforce the temporal constraints of the surgical workflow: (1) HMM-based and (2) LSTM-based pipelines. The results show that the LSTM-based approach is able to outperform the HMM-based approach and also to properly enforce the temporal constraints into the recognition process.