EgoSurgery-Tool: A Dataset of Surgical Tool and Hand Detection from Egocentric Open Surgery Videos

TL;DR

EgoSurgery-Tool is a large-scale, densely annotated dataset of egocentric open surgery videos, enabling improved detection of surgical tools and hands, which addresses existing challenges in class imbalance, occlusion, and similarity.

Contribution

The paper introduces EgoSurgery-Tool, a comprehensive dataset with over 95K annotations for surgical tools and hands, enhancing research in egocentric surgical video analysis.

Findings

Nine object detectors evaluated for effectiveness.

EgoSurgery-Tool outperforms existing datasets.

Dataset facilitates advances in surgical activity understanding.

Abstract

Surgical tool detection is a fundamental task for understanding egocentric open surgery videos. However, detecting surgical tools presents significant challenges due to their highly imbalanced class distribution, similar shapes and similar textures, and heavy occlusion. The lack of a comprehensive large-scale dataset compounds these challenges. In this paper, we introduce EgoSurgery-Tool, an extension of the existing EgoSurgery-Phase dataset, which contains real open surgery videos captured using an egocentric camera attached to the surgeon's head, along with phase annotations. EgoSurgery-Tool has been densely annotated with surgical tools and comprises over 49K surgical tool bounding boxes across 15 categories, constituting a large-scale surgical tool detection dataset. EgoSurgery-Tool also provides annotations for hand detection with over 46K hand-bounding boxes, capturing hand-object interactions that are crucial for understanding activities in egocentric open surgery. EgoSurgery-Tool is superior to existing datasets due to its larger scale, greater variety of surgical tools, more annotations, and denser scenes. We conduct a comprehensive analysis of EgoSurgery-Tool using nine popular object detectors to assess their effectiveness in both surgical tool and hand detection. The dataset will be released at https://github.com/Fujiry0/EgoSurgery.