Large language models improve Alzheimer's disease diagnosis using multi-modality data

TL;DR

This paper demonstrates that integrating large language models with multi-modality data significantly improves Alzheimer's disease diagnosis accuracy, leveraging both imaging and non-imaging patient information.

Contribution

The study introduces a novel approach that combines large language models with multi-modal data, enhancing AI's ability to utilize non-image information for AD diagnosis.

Findings

Achieved state-of-the-art results on the ADNI dataset.

Enhanced model performance by incorporating non-image data.

Showed the effectiveness of LLMs in medical diagnosis tasks.

Abstract

In diagnosing challenging conditions such as Alzheimer's disease (AD), imaging is an important reference. Non-imaging patient data such as patient information, genetic data, medication information, cognitive and memory tests also play a very important role in diagnosis. Effect. However, limited by the ability of artificial intelligence models to mine such information, most of the existing models only use multi-modal image data, and cannot make full use of non-image data. We use a currently very popular pre-trained large language model (LLM) to enhance the model's ability to utilize non-image data, and achieved SOTA results on the ADNI dataset.