Match4Annotate: Propagating Sparse Video Annotations via Implicit Neural Feature Matching

TL;DR

Match4Annotate introduces a novel neural feature matching framework that efficiently propagates annotations across video frames and videos, significantly improving scalability and accuracy in medical imaging annotation tasks.

Contribution

The paper presents a lightweight, test-time optimized neural feature matching method that unifies intra- and inter-video annotation propagation, overcoming limitations of existing approaches.

Findings

Achieves state-of-the-art inter-video propagation performance.

Outperforms traditional feature matching and segmentation baselines.

Remains competitive with specialized trackers for intra-video tasks.

Abstract

Acquiring per-frame video annotations remains a primary bottleneck for deploying computer vision in specialized domains such as medical imaging, where expert labeling is slow and costly. Label propagation offers a natural solution, yet existing approaches face fundamental limitations. Video trackers and segmentation models can propagate labels within a single sequence but require per-video initialization and cannot generalize across videos. Classic correspondence pipelines operate on detector-chosen keypoints and struggle in low-texture scenes, while dense feature matching and one-shot segmentation methods enable cross-video propagation but lack spatiotemporal smoothness and unified support for both point and mask annotations. We present Match4Annotate, a lightweight framework for both intra-video and inter-video propagation of point and mask annotations. Our method fits a SIREN-based implicit neural representation to DINOv3 features at test time, producing a continuous, high-resolution spatiotemporal feature field, and learns a smooth implicit deformation field between frame pairs to guide correspondence matching. We evaluate on three challenging clinical ultrasound datasets. Match4Annotate achieves state-of-the-art inter-video propagation, outperforming feature matching and one-shot segmentation baselines, while remaining competitive with specialized trackers for intra-video propagation. Our results show that lightweight, test-time-optimized feature matching pipelines have the potential to offer an efficient and accessible solution for scalable annotation workflows.