Where is VALDO? VAscular Lesions Detection and segmentatiOn challenge at MICCAI 2021

TL;DR

The paper reports on the MICCAI 2021 VALDO challenge, which aimed to develop automated methods for detecting and segmenting small brain lesions related to small vessel disease, highlighting variability in current approaches and potential for population-level analysis.

Contribution

It introduces a benchmark challenge for automated detection of cerebral small vessel disease markers, providing a platform for comparing methods and identifying current limitations.

Findings

Promising results for detecting enlarged perivascular spaces and microbleeds.

High variability in method performance across teams and tasks.

Limited practical utility for lacune detection at individual level.

Abstract

Imaging markers of cerebral small vessel disease provide valuable information on brain health, but their manual assessment is time-consuming and hampered by substantial intra- and interrater variability. Automated rating may benefit biomedical research, as well as clinical assessment, but diagnostic reliability of existing algorithms is unknown. Here, we present the results of the \textit{VAscular Lesions DetectiOn and Segmentation} (\textit{Where is VALDO?}) challenge that was run as a satellite event at the international conference on Medical Image Computing and Computer Aided Intervention (MICCAI) 2021. This challenge aimed to promote the development of methods for automated detection and segmentation of small and sparse imaging markers of cerebral small vessel disease, namely enlarged perivascular spaces (EPVS) (Task 1), cerebral microbleeds (Task 2) and lacunes of presumed vascular origin (Task 3) while leveraging weak and noisy labels. Overall, 12 teams participated in the challenge proposing solutions for one or more tasks (4 for Task 1 - EPVS, 9 for Task 2 - Microbleeds and 6 for Task 3 - Lacunes). Multi-cohort data was used in both training and evaluation. Results showed a large variability in performance both across teams and across tasks, with promising results notably for Task 1 - EPVS and Task 2 - Microbleeds and not practically useful results yet for Task 3 - Lacunes. It also highlighted the performance inconsistency across cases that may deter use at an individual level, while still proving useful at a population level.