Validating Deep Models for Alzheimer's 18F-FDG PET Diagnosis Across Populations: A Study with Latin American Data

TL;DR

This study evaluates the generalization of deep learning models for Alzheimer's diagnosis using 18F-FDG PET scans across North American and Latin American populations, revealing significant domain shifts and emphasizing the need for population-aware validation.

Contribution

It benchmarks convolutional and Transformer models on diverse cohorts, highlighting the impact of domain shift and key factors for improving model generalization in neuroimaging diagnostics.

Findings

High performance on ADNI dataset (up to .97 AUC)

Performance drops on Latin American cohort (down to .80 AUC)

Normalization and sampling are crucial for generalization

Abstract

Deep learning models have shown strong performance in diagnosing Alzheimer's disease (AD) using neuroimaging data, particularly 18F-FDG PET scans, with training datasets largely composed of North American cohorts such as those in the Alzheimer's Disease Neuroimaging Initiative (ADNI). However, their generalization to underrepresented populations remains underexplored. In this study, we benchmark convolutional and Transformer-based models on the ADNI dataset and assess their generalization performance on a novel Latin American clinical cohort from the FLENI Institute in Buenos Aires, Argentina. We show that while all models achieve high AUCs on ADNI (up to .96, .97), their performance drops substantially on FLENI (down to .82, .80, respectively), revealing a significant domain shift. The tested architectures demonstrated similar performance, calling into question the supposed advantages of transformers for this specific task. Through ablation studies, we identify per-image normalization and a correct sampling selection as key factors for generalization. Occlusion sensitivity analysis further reveals that models trained on ADNI, generally attend to canonical hypometabolic regions for the AD class, but focus becomes unclear for the other classes and for FLENI scans. These findings highlight the need for population-aware validation of diagnostic AI models and motivate future work on domain adaptation and cohort diversification.