LLM-MedQA: Enhancing Medical Question Answering through Case Studies in Large Language Models

TL;DR

This paper introduces a multi-agent LLM-based system that enhances medical question answering by generating similar cases, leveraging zero-shot learning with Llama3.1, resulting in significant performance improvements and better interpretability.

Contribution

It presents a novel multi-agent approach using Llama3.1 for medical QA that improves accuracy without additional training data.

Findings

7% accuracy improvement over benchmarks

Enhanced interpretability and reliability in complex queries

Effective zero-shot learning in medical question answering

Abstract

Accurate and efficient question-answering systems are essential for delivering high-quality patient care in the medical field. While Large Language Models (LLMs) have made remarkable strides across various domains, they continue to face significant challenges in medical question answering, particularly in understanding domain-specific terminologies and performing complex reasoning. These limitations undermine their effectiveness in critical medical applications. To address these issues, we propose a novel approach incorporating similar case generation within a multi-agent medical question-answering (MedQA) system. Specifically, we leverage the Llama3.1:70B model, a state-of-the-art LLM, in a multi-agent architecture to enhance performance on the MedQA dataset using zero-shot learning. Our method capitalizes on the model's inherent medical knowledge and reasoning capabilities, eliminating the need for additional training data. Experimental results show substantial performance gains over existing benchmark models, with improvements of 7% in both accuracy and F1-score across various medical QA tasks. Furthermore, we examine the model's interpretability and reliability in addressing complex medical queries. This research not only offers a robust solution for medical question answering but also establishes a foundation for broader applications of LLMs in the medical domain.