Generative Data-engine Foundation Model for Universal Few-shot 2D Vascular Image Segmentation

TL;DR

UniVG is a generative foundation model that synthesizes diverse vascular images and enables few-shot segmentation, reducing the need for extensive annotated data in 2D vascular imaging.

Contribution

The paper introduces UniVG, a novel generative model that learns vascular image compositionality and adapts with minimal data for universal few-shot segmentation.

Findings

Achieves performance comparable to fully supervised models with only 5 labeled images per task.

Develops UniVG-58K dataset with over 58,000 vascular images across five modalities.

Demonstrates robustness across 11 segmentation tasks and five imaging modalities.

Abstract

The segmentation of 2D vascular structures via deep learning holds significant clinical value but is hindered by the scarcity of annotated data, severely limiting its widespread application. Developing a universal few-shot vascular segmentation model is highly desirable, yet remains challenging due to the need for extensive training and the inherent complexities of vascular imaging. In this work, we propose UniVG (Generative Data-engine Foundation Model for Universal Few-shot 2D Vascular Image Segmentation), a novel approach that learns the compositionality of vascular images and constructing a generative foundation model for robust vascular segmentation. UniVG enables the synthesis and learning of diverse and realistic vascular images through two key innovations: 1) Compositional learning for flexible and diverse vascular synthesis: It decomposes and recombines vascular structures with varying morphological features and diverse foreground-background configurations to generate richly diverse synthetic image-label pairs. 2) Few-shot generative adaptation for transferable segmentation: It fine-tunes pre-trained models with minimal annotated data to bridge the gap between synthetic and real vascular domains, synthesizing authentic and diverse vessel images for downstream few-shot vascular segmentation learning. To support our approach, we develop UniVG-58K, a large dataset comprising 58,689 vascular images across five imaging modalities, facilitating robust large-scale generative pre-training. Extensive experiments on 11 vessel segmentation tasks cross 5 modalties (only with 5 labeled images on each task) demonstrate that UniVG achieves performance comparable to fully supervised models, significantly reducing data collection and annotation costs. All code and datasets will be made publicly available at https://github.com/XinAloha/UniVG.