You Are Your Best Teacher: Semi-Supervised Surgical Point Tracking with Cycle-Consistent Self-Distillation

TL;DR

SurgTracker is a semi-supervised method that adapts synthetic-trained surgical point trackers to real videos by using cycle consistency to filter pseudo-labels, improving robustness with minimal unlabeled data.

Contribution

The paper introduces SurgTracker, a novel semi-supervised framework employing cycle-consistent self-distillation for surgical point tracking adaptation.

Findings

Improves tracking accuracy on the STIR benchmark.

Achieves significant performance gains with only 80 unlabeled videos.

Enforces geometric consistency without multiple teacher models.

Abstract

Synthetic datasets have enabled significant progress in point tracking by providing large-scale, densely annotated supervision. However, deploying these models in real-world domains remains challenging due to domain shift and lack of labeled data-issues that are especially severe in surgical videos, where scenes exhibit complex tissue deformation, occlusion, and lighting variation. While recent approaches adapt synthetic-trained trackers to natural videos using teacher ensembles or augmentation-heavy pseudo-labeling pipelines, their effectiveness in high-shift domains like surgery remains unexplored. This work presents SurgTracker, a semi-supervised framework for adapting synthetic-trained point trackers to surgical video using filtered self-distillation. Pseudo-labels are generated online by a fixed teacher-identical in architecture and initialization to the student-and are filtered using a cycle consistency constraint to discard temporally inconsistent trajectories. This simple yet effective design enforces geometric consistency and provides stable supervision throughout training, without the computational overhead of maintaining multiple teachers. Experiments on the STIR benchmark show that SurgTracker improves tracking performance using only 80 unlabeled videos, demonstrating its potential for robust adaptation in high-shift, data-scarce domains.