Multitask Learning in Minimally Invasive Surgical Vision: A Review

TL;DR

This review discusses how multitask learning enhances surgical video analysis in minimally invasive surgery, improving understanding and efficiency by leveraging related tasks and recent machine learning advancements.

Contribution

It provides a comprehensive overview of current multitask learning systems in MIS, analyzing their benefits, limitations, and future research directions.

Findings

MTL improves surgical scene understanding accuracy

Recent trends show increased use of large models in MIS

Identifies challenges and future directions in MTL for MIS

Abstract

Minimally invasive surgery (MIS) has revolutionized many procedures and led to reduced recovery time and risk of patient injury. However, MIS poses additional complexity and burden on surgical teams. Data-driven surgical vision algorithms are thought to be key building blocks in the development of future MIS systems with improved autonomy. Recent advancements in machine learning and computer vision have led to successful applications in analyzing videos obtained from MIS with the promise of alleviating challenges in MIS videos. Surgical scene and action understanding encompasses multiple related tasks that, when solved individually, can be memory-intensive, inefficient, and fail to capture task relationships. Multitask learning (MTL), a learning paradigm that leverages information from multiple related tasks to improve performance and aid generalization, is well suited for fine-grained and high-level understanding of MIS data. This review provides a narrative overview of the current state-of-the-art MTL systems that leverage videos obtained from MIS. Beyond listing published approaches, we discuss the benefits and limitations of these MTL systems. Moreover, this manuscript presents an analysis of the literature for various application fields of MTL in MIS, including those with large models, highlighting notable trends, new directions of research, and developments.