AlzheimerRAG: Multimodal Retrieval Augmented Generation for Clinical Use Cases using PubMed articles

TL;DR

AlzheimerRAG is a multimodal retrieval-augmented generation system that leverages PubMed articles to improve clinical decision support for Alzheimer's Disease through enhanced information retrieval and synthesis.

Contribution

This paper introduces AlzheimerRAG, a novel multimodal RAG system that integrates textual and visual biomedical data for clinical use, demonstrating improved retrieval and synthesis performance.

Findings

Outperforms benchmarks like BioASQ and PubMedQA in retrieval accuracy.

Generates clinically relevant responses with low hallucination rates.

Effective in various Alzheimer's clinical scenarios.

Abstract

Recent advancements in generative AI have fostered the development of highly adept Large Language Models (LLMs) that integrate diverse data types to empower decision-making. Among these, multimodal retrieval-augmented generation (RAG) applications are promising because they combine the strengths of information retrieval and generative models, enhancing their utility across various domains, including clinical use cases. This paper introduces AlzheimerRAG, a Multimodal RAG application for clinical use cases, primarily focusing on Alzheimer's Disease case studies from PubMed articles. This application incorporates cross-modal attention fusion techniques to integrate textual and visual data processing by efficiently indexing and accessing vast amounts of biomedical literature. Our experimental results, compared to benchmarks such as BioASQ and PubMedQA, have yielded improved performance in the retrieval and synthesis of domain-specific information. We also present a case study using our multimodal RAG in various Alzheimer's clinical scenarios. We infer that AlzheimerRAG can generate responses with accuracy non-inferior to humans and with low rates of hallucination.