Finding Common Ground: Using Large Language Models to Detect Agreement in Multi-Agent Decision Conferences

TL;DR

This paper introduces a novel multi-agent system using large language models to simulate decision conferences, focusing on detecting agreement among agents to improve collaborative decision-making processes.

Contribution

It presents a new LLM-based multi-agent system for simulating decision conferences and evaluating agreement detection, advancing the use of AI in modeling complex group interactions.

Findings

LLMs reliably detect agreement in dynamic debates

Incorporating agreement detection improves debate efficiency

System simulates real-world decision conference outcomes

Abstract

Decision conferences are structured, collaborative meetings that bring together experts from various fields to address complex issues and reach a consensus on recommendations for future actions or policies. These conferences often rely on facilitated discussions to ensure productive dialogue and collective agreement. Recently, Large Language Models (LLMs) have shown significant promise in simulating real-world scenarios, particularly through collaborative multi-agent systems that mimic group interactions. In this work, we present a novel LLM-based multi-agent system designed to simulate decision conferences, specifically focusing on detecting agreement among the participant agents. To achieve this, we evaluate six distinct LLMs on two tasks: stance detection, which identifies the position an agent takes on a given issue, and stance polarity detection, which identifies the sentiment as positive, negative, or neutral. These models are further assessed within the multi-agent system to determine their effectiveness in complex simulations. Our results indicate that LLMs can reliably detect agreement even in dynamic and nuanced debates. Incorporating an agreement-detection agent within the system can also improve the efficiency of group debates and enhance the overall quality and coherence of deliberations, making them comparable to real-world decision conferences regarding outcome and decision-making. These findings demonstrate the potential for LLM-based multi-agent systems to simulate group decision-making processes. They also highlight that such systems could be instrumental in supporting decision-making with expert elicitation workshops across various domains.