Quantifying AI-to-Clinical Translation: The Algorithm-to-Outcome Concordance (AOC) Framework

TL;DR

The paper introduces the Algorithm-to-Outcome Concordance (AOC) framework to quantitatively assess the translation of AI biomarkers into clinical outcomes, addressing the gap between in-silico performance and real-world efficacy.

Contribution

It presents a novel AOC framework that combines model accuracy, clinical correlation, and trial heterogeneity, validated across multiple immunotherapy trials and cohorts.

Findings

AOC scores correlate with trial success (ORR >15%)

External validation shows unstable algorithm-outcome correlation

AOC enables pre-trial risk assessment and adaptive design

Abstract

Background: Despite high in-silico performance (AUC >0.80), 85% of AI cancer biomarkers fail clinical translation, exposing a critical algorithm-to-outcome gap. Methods: We introduce the Algorithm-to-Outcome Concordance (AOC) framework, integrating model accuracy (AUC), clinical correlation (Corr), and trial heterogeneity. We validated AOC across 6 neoantigen vaccine trials (2017-2025) and 3 independent melanoma immunotherapy cohorts (n=188 patients). Results: AOC ranged 0.18-0.79 across trials, with failed trials (ORR <15%) showing AOC <0.40. External validation revealed unstable algorithm-outcome correlation (C-index: 0.49-0.61, p>0.05), demonstrating the necessity of explicit concordance assessment. Conclusions: AOC provides a quantitative framework for pre-trial risk assessment and adaptive trial design. Prospective validation is underway in KEYNOTE-942 extension studies.