Scaling up self-supervised learning for improved surgical foundation models

TL;DR

This paper introduces SurgeNetXL, a large-scale surgical foundation model trained on over 4.7 million video frames, achieving state-of-the-art performance across multiple surgical vision tasks and providing insights into scaling datasets and model architectures.

Contribution

The study presents SurgeNetXL, a novel surgical foundation model trained on the largest surgical dataset to date, setting new benchmarks and offering key insights into scaling pretraining for surgical computer vision.

Findings

SurgeNetXL outperforms previous models by 2.4-12.6% across tasks.

SurgeNetXL surpasses ImageNet-based variants by 1.6-14.4%.

Scaling datasets and training duration improves model performance.

Abstract

Foundation models have revolutionized computer vision by achieving vastly superior performance across diverse tasks through large-scale pretraining on extensive datasets. However, their application in surgical computer vision has been limited. This study addresses this gap by introducing SurgeNetXL, a novel surgical foundation model that sets a new benchmark in surgical computer vision. Trained on the largest reported surgical dataset to date, comprising over 4.7 million video frames, SurgeNetXL achieves consistent top-tier performance across six datasets spanning four surgical procedures and three tasks, including semantic segmentation, phase recognition, and critical view of safety (CVS) classification. Compared with the best-performing surgical foundation models, SurgeNetXL shows mean improvements of 2.4, 9.0, and 12.6 percent for semantic segmentation, phase recognition, and CVS classification, respectively. Additionally, SurgeNetXL outperforms the best-performing ImageNet-based variants by 14.4, 4.0, and 1.6 percent in the respective tasks. In addition to advancing model performance, this study provides key insights into scaling pretraining datasets, extending training durations, and optimizing model architectures specifically for surgical computer vision. These findings pave the way for improved generalizability and robustness in data-scarce scenarios, offering a comprehensive framework for future research in this domain. All models and a subset of the SurgeNetXL dataset, including over 2 million video frames, are publicly available at: https://github.com/TimJaspers0801/SurgeNet.