Fine-Tuning LLMs for Reliable Medical Question-Answering Services

TL;DR

This paper demonstrates how fine-tuning large language models with specialized techniques can significantly enhance the accuracy and reliability of medical question-answering systems, supporting better healthcare decision-making.

Contribution

It introduces novel fine-tuning methods like rsDoRA+ and ReRAG for improving LLM performance in medical QA tasks, with a focus on accuracy and efficiency.

Findings

Improved medical answer accuracy with fine-tuned LLMs

Enhanced efficiency through rsDoRA+ optimization

Refined responses via retrieval on demand and question rewriting

Abstract

We present an advanced approach to medical question-answering (QA) services, using fine-tuned Large Language Models (LLMs) to improve the accuracy and reliability of healthcare information. Our study focuses on optimizing models like LLaMA-2 and Mistral, which have shown great promise in delivering precise, reliable medical answers. By leveraging comprehensive datasets, we applied fine-tuning techniques such as rsDoRA+ and ReRAG. rsDoRA+ enhances model performance through a combination of decomposed model weights, varied learning rates for low-rank matrices, and rank stabilization, leading to improved efficiency. ReRAG, which integrates retrieval on demand and question rewriting, further refines the accuracy of the responses. This approach enables healthcare providers to access fast, dependable information, aiding in more efficient decision-making and fostering greater patient trust. Our work highlights the potential of fine-tuned LLMs to significantly improve the quality and accessibility of medical information services, ultimately contributing to better healthcare outcomes for all.