a2z-1 for Multi-Disease Detection in Abdomen-Pelvis CT: External Validation and Performance Analysis Across 21 Conditions

TL;DR

This study evaluates the a2z-1 AI model for detecting 21 conditions in abdomen-pelvis CT scans, demonstrating high accuracy, robustness across diverse settings, and potential to assist radiologists in clinical workflows.

Contribution

The paper provides a large-scale external validation of a2z-1, showing its generalizability and consistent performance across multiple conditions and patient subgroups.

Findings

Average AUC of 0.931 across 21 conditions

External validation confirms AUC 0.923 in different health systems

Model identified overlooked findings, aiding quality assurance

Abstract

We present a comprehensive evaluation of a2z-1, an artificial intelligence (AI) model designed to analyze abdomen-pelvis CT scans for 21 time-sensitive and actionable findings. Our study focuses on rigorous assessment of the model's performance and generalizability. Large-scale retrospective analysis demonstrates an average AUC of 0.931 across 21 conditions. External validation across two distinct health systems confirms consistent performance (AUC 0.923), establishing generalizability to different evaluation scenarios, with notable performance in critical findings such as small bowel obstruction (AUC 0.958) and acute pancreatitis (AUC 0.961). Subgroup analysis shows consistent accuracy across patient sex, age groups, and varied imaging protocols, including different slice thicknesses and contrast administration types. Comparison of high-confidence model outputs to radiologist reports reveals instances where a2z-1 identified overlooked findings, suggesting potential for quality assurance applications.