Statistical testing on generative AI anomaly detection tools in Alzheimer's Disease diagnosis

TL;DR

This paper introduces a statistically rigorous approach using selective inference to evaluate generative AI tools for Alzheimer's diagnosis, addressing reliability issues and aiding clinical decision-making.

Contribution

It develops a novel selective inference framework for validating generative AI models in Alzheimer's detection, controlling false discoveries and improving reliability.

Findings

Selective inference controls false discovery rate effectively.

Generative AI shows promise in early Alzheimer's detection.

Method outperforms traditional statistical tests in reliability.

Abstract

Alzheimer's Disease is challenging to diagnose due to our limited understanding of its mechanism and large heterogeneity among patients. Neurodegeneration is studied widely as a biomarker for clinical diagnosis, which can be measured from time series MRI progression. On the other hand, generative AI has shown promise in anomaly detection in medical imaging and used for tasks including tumor detection. However, testing the reliability of such data-driven methods is non-trivial due to the issue of double-dipping in hypothesis testing. In this work, we propose to solve this issue with selective inference and develop a reliable generative AI method for Alzheimer's prediction. We show that compared to traditional statistical methods with highly inflated p-values, selective inference successfully controls the false discovery rate under the desired alpha level while retaining statistical power. In practice, our pipeline could assist clinicians in Alzheimer's diagnosis and early intervention.