Leveraging Multimodal Models for Enhanced Neuroimaging Diagnostics in Alzheimer's Disease

TL;DR

This paper introduces a multimodal approach that combines synthetic report generation and image-to-text translation to improve neuroimaging diagnostics for Alzheimer's disease, leveraging large language and vision-language models.

Contribution

It presents a novel method for generating diagnostic reports from neuroimages using synthetic data and pre-trained models, addressing data scarcity in Alzheimer's diagnostics.

Findings

Achieved BLEU-4 score of 0.1827

ROUGE-L score of 0.3719

METEOR score of 0.4163

Abstract

The rapid advancements in Large Language Models (LLMs) and Vision-Language Models (VLMs) have shown great potential in medical diagnostics, particularly in radiology, where datasets such as X-rays are paired with human-generated diagnostic reports. However, a significant research gap exists in the neuroimaging field, especially for conditions such as Alzheimer's disease, due to the lack of comprehensive diagnostic reports that can be utilized for model fine-tuning. This paper addresses this gap by generating synthetic diagnostic reports using GPT-4o-mini on structured data from the OASIS-4 dataset, which comprises 663 patients. Using the synthetic reports as ground truth for training and validation, we then generated neurological reports directly from the images in the dataset leveraging the pre-trained BiomedCLIP and T5 models. Our proposed method achieved a BLEU-4 score of 0.1827, ROUGE-L score of 0.3719, and METEOR score of 0.4163, revealing its potential in generating clinically relevant and accurate diagnostic reports.