Robust Surgical Tools Detection in Endoscopic Videos with Noisy Data

TL;DR

This paper presents a new semi-supervised learning approach with active learning and self-training to improve surgical tool detection in noisy, poorly annotated endoscopic video data, achieving high accuracy and outperforming existing methods.

Contribution

It introduces a systematic methodology combining active learning and self-training with class weighting to develop robust surgical tool detection models from noisy datasets.

Findings

Achieved an average F1-score of 85.88% with class weights.

Outperformed existing approaches in noisy label scenarios.

Demonstrated effectiveness of the proposed semi-supervised framework.

Abstract

Over the past few years, surgical data science has attracted substantial interest from the machine learning (ML) community. Various studies have demonstrated the efficacy of emerging ML techniques in analysing surgical data, particularly recordings of procedures, for digitizing clinical and non-clinical functions like preoperative planning, context-aware decision-making, and operating skill assessment. However, this field is still in its infancy and lacks representative, well-annotated datasets for training robust models in intermediate ML tasks. Also, existing datasets suffer from inaccurate labels, hindering the development of reliable models. In this paper, we propose a systematic methodology for developing robust models for surgical tool detection using noisy data. Our methodology introduces two key innovations: (1) an intelligent active learning strategy for minimal dataset identification and label correction by human experts; and (2) an assembling strategy for a student-teacher model-based self-training framework to achieve the robust classification of 14 surgical tools in a semi-supervised fashion. Furthermore, we employ weighted data loaders to handle difficult class labels and address class imbalance issues. The proposed methodology achieves an average F1-score of 85.88\% for the ensemble model-based self-training with class weights, and 80.88\% without class weights for noisy labels. Also, our proposed method significantly outperforms existing approaches, which effectively demonstrates its effectiveness.