RAG-based Architectures for Drug Side Effect Retrieval in LLMs

TL;DR

This paper introduces RAG-based architectures, including GraphRAG, that significantly improve the accuracy of drug side effect retrieval in LLMs, addressing limitations like hallucinations and domain knowledge gaps.

Contribution

The paper presents novel RAG and GraphRAG architectures that incorporate domain-specific drug side effect knowledge into LLMs for enhanced pharmacovigilance.

Findings

GraphRAG achieves near-perfect accuracy in drug side effect retrieval.

Extensive evaluation on 19,520 associations demonstrates high scalability and reliability.

Framework significantly advances LLM applications in pharmacovigilance.

Abstract

Drug side effects are a major global health concern, necessitating advanced methods for their accurate detection and analysis. While Large Language Models (LLMs) offer promising conversational interfaces, their inherent limitations, including reliance on black-box training data, susceptibility to hallucinations, and lack of domain-specific knowledge, hinder their reliability in specialized fields like pharmacovigilance. To address this gap, we propose two architectures: Retrieval-Augmented Generation (RAG) and GraphRAG, which integrate comprehensive drug side effect knowledge into a Llama 3 8B language model. Through extensive evaluations on 19,520 drug side effect associations (covering 976 drugs and 3,851 side effect terms), our results demonstrate that GraphRAG achieves near-perfect accuracy in drug side effect retrieval. This framework offers a highly accurate and scalable solution, signifying a significant advancement in leveraging LLMs for critical pharmacovigilance applications.