Medico: Towards Hallucination Detection and Correction with Multi-source Evidence Fusion

TL;DR

Medico is an automated framework that detects and corrects hallucinations in LLM outputs by fusing multiple sources of evidence, significantly improving factual accuracy and providing explanations.

Contribution

It introduces a novel multi-source evidence fusion approach for automatic hallucination detection and correction in LLMs, enhancing factual reliability.

Findings

High accuracy in evidence retrieval (HR@5 0.964)

Effective hallucination detection (F1 0.927-0.951)

Successful content correction (approval rate 0.973-0.979)

Abstract

As we all know, hallucinations prevail in Large Language Models (LLMs), where the generated content is coherent but factually incorrect, which inflicts a heavy blow on the widespread application of LLMs. Previous studies have shown that LLMs could confidently state non-existent facts rather than answering ``I don't know''. Therefore, it is necessary to resort to external knowledge to detect and correct the hallucinated content. Since manual detection and correction of factual errors is labor-intensive, developing an automatic end-to-end hallucination-checking approach is indeed a needful thing. To this end, we present Medico, a Multi-source evidence fusion enhanced hallucination detection and correction framework. It fuses diverse evidence from multiple sources, detects whether the generated content contains factual errors, provides the rationale behind the judgment, and iteratively revises the hallucinated content. Experimental results on evidence retrieval (0.964 HR@5, 0.908 MRR@5), hallucination detection (0.927-0.951 F1), and hallucination correction (0.973-0.979 approval rate) manifest the great potential of Medico. A video demo of Medico can be found at https://youtu.be/RtsO6CSesBI.