Benchmarking Retinal Blood Vessel Segmentation Models for Cross-Dataset and Cross-Disease Generalization

TL;DR

This study benchmarks various retinal blood vessel segmentation models on a large, diverse dataset to evaluate their generalization across diseases, image qualities, and domain shifts, providing practical guidance for clinical applications.

Contribution

It offers a comprehensive benchmark of segmentation models on the largest dataset to date, analyzing their robustness across conditions and image qualities, and highlighting the importance of dataset quality over model complexity.

Findings

Basic architectures like U-Net perform comparably to advanced models with sufficient data.

Model transferability across disease-induced domain shifts is generally effective.

Image quality significantly impacts segmentation performance.

Abstract

Retinal blood vessel segmentation can extract clinically relevant information from fundus images. As manual tracing is cumbersome, algorithms based on Convolution Neural Networks have been developed. Such studies have used small publicly available datasets for training and measuring performance, running the risk of overfitting. Here, we provide a rigorous benchmark for various architectural and training choices commonly used in the literature on the largest dataset published to date. We train and evaluate five published models on the publicly available FIVES fundus image dataset, which exceeds previous ones in size and quality and which contains also images from common ophthalmological conditions (diabetic retinopathy, age-related macular degeneration, glaucoma). We compare the performance of different model architectures across different loss functions, levels of image qualitiy and ophthalmological conditions and assess their ability to perform well in the face of disease-induced domain shifts. Given sufficient training data, basic architectures such as U-Net perform just as well as more advanced ones, and transfer across disease-induced domain shifts typically works well for most architectures. However, we find that image quality is a key factor determining segmentation outcomes. When optimizing for segmentation performance, investing into a well curated dataset to train a standard architecture yields better results than tuning a sophisticated architecture on a smaller dataset or one with lower image quality. We distilled the utility of architectural advances in terms of their clinical relevance therefore providing practical guidance for model choices depending on the circumstances of the clinical setting