Enhancing Multi-Agent Consensus through Third-Party LLM Integration: Analyzing Uncertainty and Mitigating Hallucinations in Large Language Models

TL;DR

This paper proposes a multi-agent system integrating third-party LLMs to improve consensus, reduce hallucinations, and enhance reasoning in complex tasks, validated by experiments on arithmetic datasets.

Contribution

It introduces a novel method using third-party LLMs for attention adjustment based on uncertainty, advancing multi-agent consensus and hallucination mitigation.

Findings

Outperforms traditional multi-agent baselines on arithmetic datasets

Effectively reduces hallucinations in complex reasoning tasks

Enhances consensus accuracy through uncertainty-based attention adjustment

Abstract

Large Language Models (LLMs) still face challenges when dealing with complex reasoning tasks, often resulting in hallucinations, which limit the practical application of LLMs. To alleviate this issue, this paper proposes a new method that integrates different LLMs to expand the knowledge boundary, reduce dependence on a single model, and promote in-depth debate among agents. The main contributions include: 1) Introducing third-party LLMs to adjust the attention weights of agents through uncertainty estimation and confidence analysis, optimizing consensus formation in multi-agent systems; 2) Experiments on arithmetic datasets have validated the effectiveness of the method, surpassing traditional multi-agent baselines. This research provides a new perspective for large models to alleviate hallucination phenomena when dealing with complex tasks.