LesionLocator: Zero-Shot Universal Tumor Segmentation and Tracking in 3D Whole-Body Imaging

TL;DR

LesionLocator is a novel zero-shot framework for 3D medical image lesion segmentation and tracking, achieving human-level performance and setting new benchmarks with a large-scale dataset and open-source tools.

Contribution

It introduces the first end-to-end 4D lesion tracking model capable of zero-shot generalization and provides the largest synthetic dataset for this task.

Findings

Outperforms existing models by nearly 10 dice points in segmentation

Achieves state-of-the-art lesion tracking accuracy

Reaches human-level performance in lesion segmentation

Abstract

In this work, we present LesionLocator, a framework for zero-shot longitudinal lesion tracking and segmentation in 3D medical imaging, establishing the first end-to-end model capable of 4D tracking with dense spatial prompts. Our model leverages an extensive dataset of 23,262 annotated medical scans, as well as synthesized longitudinal data across diverse lesion types. The diversity and scale of our dataset significantly enhances model generalizability to real-world medical imaging challenges and addresses key limitations in longitudinal data availability. LesionLocator outperforms all existing promptable models in lesion segmentation by nearly 10 dice points, reaching human-level performance, and achieves state-of-the-art results in lesion tracking, with superior lesion retrieval and segmentation accuracy. LesionLocator not only sets a new benchmark in universal promptable lesion segmentation and automated longitudinal lesion tracking but also provides the first open-access solution of its kind, releasing our synthetic 4D dataset and model to the community, empowering future advancements in medical imaging. Code is available at: www.github.com/MIC-DKFZ/LesionLocator