Mitigating Hallucinations in Healthcare LLMs with Granular Fact-Checking and Domain-Specific Adaptation

TL;DR

This paper introduces a fact-checking system and a domain-specific summarization model to reduce hallucinations in healthcare language models, improving reliability and accuracy in critical medical applications.

Contribution

It presents a novel independent fact-checking module combined with a fine-tuned summarization model to enhance factual accuracy in healthcare LLM outputs.

Findings

Fact-checking module achieves high precision and recall (around 0.89 and 0.82).

Summarization model attains ROUGE-1 of 0.58 and BERTScore of 0.91.

The approach significantly reduces hallucination rates in healthcare LLMs.

Abstract

In healthcare, it is essential for any LLM-generated output to be reliable and accurate, particularly in cases involving decision-making and patient safety. However, the outputs are often unreliable in such critical areas due to the risk of hallucinated outputs from the LLMs. To address this issue, we propose a fact-checking module that operates independently of any LLM, along with a domain-specific summarization model designed to minimize hallucination rates. Our model is fine-tuned using Low-Rank Adaptation (LoRa) on the MIMIC III dataset and is paired with the fact-checking module, which uses numerical tests for correctness and logical checks at a granular level through discrete logic in natural language processing (NLP) to validate facts against electronic health records (EHRs). We trained the LLM model on the full MIMIC-III dataset. For evaluation of the fact-checking module, we sampled 104 summaries, extracted them into 3,786 propositions, and used these as facts. The fact-checking module achieves a precision of 0.8904, a recall of 0.8234, and an F1-score of 0.8556. Additionally, the LLM summary model achieves a ROUGE-1 score of 0.5797 and a BERTScore of 0.9120 for summary quality.