Nested ResNet: A Vision-Based Method for Detecting the Sensing Area of a Drop-in Gamma Probe

TL;DR

This paper introduces a Nested ResNet-based deep learning framework that significantly improves the accuracy of visual sensing area prediction for drop-in gamma probes in robotic-assisted surgery, enabling better localization and feedback.

Contribution

The paper presents a novel three-branch deep learning architecture incorporating stereo images, depth estimation, and orientation guidance to enhance sensing area prediction accuracy.

Findings

22.10% decrease in 2D mean error

41.67% reduction in 3D mean error

Superior performance over previous methods

Abstract

Purpose: Drop-in gamma probes are widely used in robotic-assisted minimally invasive surgery (RAMIS) for lymph node detection. However, these devices only provide audio feedback on signal intensity, lacking the visual feedback necessary for precise localisation. Previous work attempted to predict the sensing area location using laparoscopic images, but the prediction accuracy was unsatisfactory. Improvements are needed in the deep learning-based regression approach. Methods: We introduce a three-branch deep learning framework to predict the sensing area of the probe. Specifically, we utilise the stereo laparoscopic images as input for the main branch and develop a Nested ResNet architecture. The framework also incorporates depth estimation via transfer learning and orientation guidance through probe axis sampling. The combined features from each branch enhanced the accuracy of the prediction. Results: Our approach has been evaluated on a publicly available dataset, demonstrating superior performance over previous methods. In particular, our method resulted in a 22.10\% decrease in 2D mean error and a 41.67\% reduction in 3D mean error. Additionally, qualitative comparisons further demonstrated the improved precision of our approach. Conclusion: With extensive evaluation, our solution significantly enhances the accuracy and reliability of sensing area predictions. This advancement enables visual feedback during the use of the drop-in gamma probe in surgery, providing surgeons with more accurate and reliable localisation.}