Tracking Everything in Robotic-Assisted Surgery

TL;DR

This paper introduces a new surgical tracking dataset and evaluates existing TAP-based algorithms, revealing their limitations in complex surgical scenarios, and proposes SurgMotion to improve tracking accuracy in robotic-assisted surgery videos.

Contribution

The paper provides a comprehensive surgical tracking dataset and introduces SurgMotion, a novel method that enhances tracking performance in challenging surgical conditions.

Findings

TAP-based algorithms struggle with fast motion, occlusions, and motion blur in surgical videos.

SurgMotion significantly outperforms TAP-based methods in instrument tracking.

The new dataset enables benchmarking and development of more robust surgical tracking algorithms.

Abstract

Accurate tracking of tissues and instruments in videos is crucial for Robotic-Assisted Minimally Invasive Surgery (RAMIS), as it enables the robot to comprehend the surgical scene with precise locations and interactions of tissues and tools. Traditional keypoint-based sparse tracking is limited by featured points, while flow-based dense two-view matching suffers from long-term drifts. Recently, the Tracking Any Point (TAP) algorithm was proposed to overcome these limitations and achieve dense accurate long-term tracking. However, its efficacy in surgical scenarios remains untested, largely due to the lack of a comprehensive surgical tracking dataset for evaluation. To address this gap, we introduce a new annotated surgical tracking dataset for benchmarking tracking methods for surgical scenarios, comprising real-world surgical videos with complex tissue and instrument motions. We extensively evaluate state-of-the-art (SOTA) TAP-based algorithms on this dataset and reveal their limitations in challenging surgical scenarios, including fast instrument motion, severe occlusions, and motion blur, etc. Furthermore, we propose a new tracking method, namely SurgMotion, to solve the challenges and further improve the tracking performance. Our proposed method outperforms most TAP-based algorithms in surgical instruments tracking, and especially demonstrates significant improvements over baselines in challenging medical videos. Our code and dataset are available at https://github.com/zhanbh1019/SurgicalMotion.