Fine-tuning Segment Anything for Real-Time Tumor Tracking in Cine-MRI

TL;DR

This paper presents a real-time tumor tracking method in cine-MRI using fine-tuned foundation models, achieving high accuracy under data scarcity and strict time constraints, with potential for MRI-guided radiotherapy.

Contribution

The work demonstrates the effective fine-tuning of SAM 2.1 for real-time tumor segmentation in cine-MRI with minimal data and strict runtime limits.

Findings

Achieved a Dice score of 0.8794 on the test set.

Selected SAM-based segmentation for its real-time performance.

Ranked 6th in the TrackRAD2025 challenge.

Abstract

In this work, we address the TrackRAD2025 challenge of real-time tumor tracking in cine-MRI sequences of the thoracic and abdominal regions under strong data scarcity constraints. Two complementary strategies were explored: (i) unsupervised registration with the IMPACT similarity metric and (ii) foundation model-based segmentation leveraging SAM 2.1 and its recent variants through prompt-based interaction. Due to the one-second runtime constraint, the SAM-based method was ultimately selected. The final configuration used SAM2.1 b+ with mask-based prompts from the first annotated slice, fine-tuned solely on the small labeled subset from TrackRAD2025. Training was configured to minimize overfitting, using 1024x1024 patches (batch size 1), standard augmentations, and a balanced Dice + IoU loss. A low uniform learning rate (0.0001) was applied to all modules (prompt encoder, decoder, Hiera backbone) to preserve generalization while adapting to annotator-specific styles. Training lasted 300 epochs (~12h on RTX A6000, 48GB). The same inference strategy was consistently applied across all anatomical sites and MRI field strengths. Test-time augmentation was considered but ultimately discarded due to negligible performance gains. The final model was selected based on the highest Dice Similarity Coefficient achieved on the validation set after fine-tuning. On the hidden test set, the model reached a Dice score of 0.8794, ranking 6th overall in the TrackRAD2025 challenge. These results highlight the strong potential of foundation models for accurate and real-time tumor tracking in MRI-guided radiotherapy.