Assessing Pancreatic Ductal Adenocarcinoma Vascular Invasion: the PDACVI Benchmark

TL;DR

This paper introduces a new benchmark dataset and challenge for AI-based assessment of vascular invasion in pancreatic cancer, emphasizing uncertainty modeling and expert disagreement.

Contribution

It provides the CURVAS-PDACVI dataset with multiple expert annotations and proposes a multi-metric evaluation framework for more reliable AI assessment.

Findings

State-of-the-art methods vary in performance at tumor-vessel interfaces.

Probabilistic models better handle ambiguous cases and expert disagreement.

Volumetric overlap alone is insufficient for clinical utility.

Abstract

Surgical resection remains the only potentially curative treatment for pancreatic ductal adenocarcinoma (PDAC), and eligibility depends on accurate assessment of vascular invasion (VI), i.e., tumor extension into adjacent critical vessels. Despite its importance for preoperative staging and surgical planning, computational VI assessment remains underexplored. Two major challenges are the lack of public datasets and the diagnostic ambiguity at the tumor-vessel interface, which leads to substantial inter-rater variability even among expert radiologists. To address these limitations, we introduce the CURVAS-PDACVI Dataset and Challenge, an open benchmark for uncertainty-aware AI in PDAC staging based on a densely annotated dataset with five independent expert annotations per scan. We also propose a multi-metric evaluation framework that extends beyond spatial overlap to include probabilistic calibration and VI assessment. Evaluation of six state-of-the-art methods shows that strong global volumetric overlap does not necessarily translate into reliable performance at clinically critical tumor-vessel interfaces. In particular, methods optimized for binary segmentation perform competitively on average overlap metrics, but often degrade in high-complexity cases with low expert consensus, either collapsing in volume or overextending at uncertain boundaries. In contrast, methods that model inter-rater disagreement produce better calibrated probabilistic maps and show greater robustness in these ambiguous cases. The benchmark highlights the limitations of volumetric accuracy as a proxy for localized surgical utility, motivating uncertainty-aware probabilistic models for preoperative decision-making.