A Standardized Radiograph-Agnostic Framework and Platform For Evaluating AI Radiological Systems

TL;DR

This paper introduces a standardized, radiograph-agnostic platform and framework designed to evaluate AI radiological systems' ability to generalize across diverse populations and imaging conditions, addressing current benchmarking challenges.

Contribution

It presents a novel platform and framework that enable comprehensive assessment of AI radiology solutions' generalization capabilities across various demographic and geographic factors.

Findings

Framework facilitates cross-population evaluation

Supports diverse radiograph datasets

Enhances benchmarking standards for AI radiology

Abstract

Radiology has been essential to accurately diagnosing diseases and assessing responses to treatment. The challenge however lies in the shortage of radiologists globally. As a response to this, a number of Artificial Intelligence solutions are being developed. The challenge Artificial Intelligence radiological solutions however face is the lack of a benchmarking and evaluation standard, and the difficulties of collecting diverse data to truly assess the ability of such systems to generalise and properly handle edge cases. We are proposing a radiograph-agnostic platform and framework that would allow any Artificial Intelligence radiological solution to be assessed on its ability to generalise across diverse geographical location, gender and age groups.