Development and evaluation of CADe systems in low-prevalence setting: The RARE25 challenge for early detection of Barrett's neoplasia

TL;DR

The RARE25 challenge introduces a large-scale, prevalence-aware benchmark for evaluating computer-aided detection systems in early Barrett's neoplasia, emphasizing real-world prevalence and clinical utility.

Contribution

It provides a public dataset and evaluation framework to develop and assess CADe systems that are robust to prevalence shifts in clinical surveillance.

Findings

Several methods achieved strong discriminative performance.

Positive predictive values remained low, indicating detection difficulty in low-prevalence settings.

All methods relied on fully supervised classification, lacking prevalence-agnostic approaches.

Abstract

Computer-aided detection (CADe) of early neoplasia in Barrett's esophagus is a low-prevalence surveillance problem in which clinically relevant findings are rare. Although many CADe systems report strong performance on balanced or enriched datasets, their behavior under realistic prevalence remains insufficiently characterized. The RARE25 challenge addresses this gap by introducing a large-scale, prevalence-aware benchmark for neoplasia detection. It includes a public training set and a hidden test set reflecting real-world incidence. Methods were evaluated using operating-point-specific metrics emphasizing high sensitivity and accounting for prevalence. Eleven teams from seven countries submitted approaches using diverse architectures, pretraining, ensembling, and calibration strategies. While several methods achieved strong discriminative performance, positive predictive values remained low, highlighting the difficulty of low-prevalence detection and the risk of overestimating clinical utility when prevalence is ignored. All methods relied on fully supervised classification despite the dominance of normal findings, indicating a lack of prevalence-agnostic approaches such as anomaly detection or one-class learning. By releasing a public dataset and a reproducible evaluation framework, RARE25 aims to support the development of CADe systems robust to prevalence shift and suitable for clinical surveillance workflows.