Requirements for Quality Assurance of AI Models for Early Detection of Lung Cancer

TL;DR

This paper emphasizes the importance of systematic quality assurance for AI models in early lung cancer detection, proposing standardized testing and continuous updates to ensure reliable performance and regulatory compliance.

Contribution

It introduces a validated reference dataset and a framework for ongoing quality assurance, addressing regulatory gaps for self-learning AI algorithms in lung cancer screening.

Findings

Proposes a standardized quality assessment framework.

Highlights the need for continuous updates reflecting demographic and technological changes.

Addresses regulatory challenges for AI in medical diagnostics.

Abstract

Lung cancer is the second most common cancer and the leading cause of cancer-related deaths worldwide. Survival largely depends on tumor stage at diagnosis, and early detection with low-dose CT can significantly reduce mortality in high-risk patients. AI can improve the detection, measurement, and characterization of pulmonary nodules while reducing assessment time. However, the training data, functionality, and performance of available AI systems vary considerably, complicating software selection and regulatory evaluation. Manufacturers must specify intended use and provide test statistics, but they can choose their training and test data, limiting standardization and comparability. Under the EU AI Act, consistent quality assurance is required for AI-based nodule detection, measurement, and characterization. This position paper proposes systematic quality assurance grounded in a validated reference dataset, including real screening cases plus phantom data to verify volume and growth rate measurements. Regular updates shall reflect demographic shifts and technological advances, ensuring ongoing relevance. Consequently, ongoing AI quality assurance is vital. Regulatory challenges are also adressed. While the MDR and the EU AI Act set baseline requirements, they do not adequately address self-learning algorithms or their updates. A standardized, transparent quality assessment - based on sensitivity, specificity, and volumetric accuracy - enables an objective evaluation of each AI solution's strengths and weaknesses. Establishing clear testing criteria and systematically using updated reference data lay the groundwork for comparable performance metrics, informing tenders, guidelines, and recommendations.