Mitigating Hallucination on Hallucination in RAG via Ensemble Voting

TL;DR

This paper introduces VOTE-RAG, a simple, training-free ensemble voting framework that effectively reduces hallucinations in Retrieval-Augmented Generation by combining multiple retrieval and response votes.

Contribution

VOTE-RAG is a novel, parallelizable, and training-free ensemble voting method that mitigates hallucinations in RAG models more efficiently than complex frameworks.

Findings

VOTE-RAG achieves comparable or better performance than existing methods.

The framework is fully parallelizable and avoids problem drift.

VOTE-RAG effectively reduces hallucinations in six benchmark datasets.

Abstract

Retrieval-Augmented Generation (RAG) aims to reduce hallucinations in Large Language Models (LLMs) by integrating external knowledge. However, RAG introduces a critical challenge: hallucination on hallucination," where flawed retrieval results mislead the generation model, leading to compounded hallucinations. To address this issue, we propose VOTE-RAG, a novel, training-free framework with a two-stage structure and efficient, parallelizable voting mechanisms. VOTE-RAG includes: (1) Retrieval Voting, where multiple agents generate diverse queries in parallel and aggregate all retrieved documents; (2) Response Voting, where multiple agents independently generate answers based on the aggregated documents, with the final output determined by majority vote. We conduct comparative experiments on six benchmark datasets. Our results show that VOTE-RAG achieves performance comparable to or surpassing more complex frameworks. Additionally, VOTE-RAG features a simpler architecture, is fully parallelizable, and avoids the problem drift" risk. Our work demonstrates that simple, reliable ensemble voting is a superior and more efficient method for mitigating RAG hallucinations.