Towards Detecting LLMs Hallucination via Markov Chain-based Multi-agent Debate Framework

TL;DR

This paper introduces a Markov Chain-based multi-agent debate framework to improve the detection of hallucinations in large language models, integrating fact-checking and verification for more accurate results.

Contribution

It presents a novel multi-agent debate approach using Markov Chains to enhance hallucination detection in LLMs, addressing limitations of previous methods.

Findings

Significant improvement over baselines in hallucination detection accuracy

Effective integration of claim detection, evidence retrieval, and verification

Validated across three generative tasks

Abstract

The advent of large language models (LLMs) has facilitated the development of natural language text generation. It also poses unprecedented challenges, with content hallucination emerging as a significant concern. Existing solutions often involve expensive and complex interventions during the training process. Moreover, some approaches emphasize problem disassembly while neglecting the crucial validation process, leading to performance degradation or limited applications. To overcome these limitations, we propose a Markov Chain-based multi-agent debate verification framework to enhance hallucination detection accuracy in concise claims. Our method integrates the fact-checking process, including claim detection, evidence retrieval, and multi-agent verification. In the verification stage, we deploy multiple agents through flexible Markov Chain-based debates to validate individual claims, ensuring meticulous verification outcomes. Experimental results across three generative tasks demonstrate that our approach achieves significant improvements over baselines.