PitVis-2023 Challenge: Workflow Recognition in videos of Endoscopic Pituitary Surgery

TL;DR

The PitVis-2023 Challenge advances automated recognition of surgical steps and instruments in endoscopic pituitary surgery videos, demonstrating the effectiveness of multi-task spatio-temporal models and providing a new benchmark dataset for the field.

Contribution

This work introduces a new dataset and challenge for workflow recognition in pituitary surgery videos, highlighting the benefits of multi-task and spatio-temporal deep learning models.

Findings

Top models improved macro-F1 scores by over 50% in step recognition.

Multi-task and spatio-temporal models outperform single-task spatial models.

The dataset and benchmark facilitate further research in minimally invasive surgery recognition.

Abstract

The field of computer vision applied to videos of minimally invasive surgery is ever-growing. Workflow recognition pertains to the automated recognition of various aspects of a surgery: including which surgical steps are performed; and which surgical instruments are used. This information can later be used to assist clinicians when learning the surgery; during live surgery; and when writing operation notes. The Pituitary Vision (PitVis) 2023 Challenge tasks the community to step and instrument recognition in videos of endoscopic pituitary surgery. This is a unique task when compared to other minimally invasive surgeries due to the smaller working space, which limits and distorts vision; and higher frequency of instrument and step switching, which requires more precise model predictions. Participants were provided with 25-videos, with results presented at the MICCAI-2023 conference as part of the Endoscopic Vision 2023 Challenge in Vancouver, Canada, on 08-Oct-2023. There were 18-submissions from 9-teams across 6-countries, using a variety of deep learning models. A commonality between the top performing models was incorporating spatio-temporal and multi-task methods, with greater than 50% and 10% macro-F1-score improvement over purely spacial single-task models in step and instrument recognition respectively. The PitVis-2023 Challenge therefore demonstrates state-of-the-art computer vision models in minimally invasive surgery are transferable to a new dataset, with surgery specific techniques used to enhance performance, progressing the field further. Benchmark results are provided in the paper, and the dataset is publicly available at: https://doi.org/10.5522/04/26531686.