An Electoral Approach to Diversify LLM-based Multi-Agent Collective Decision-Making

TL;DR

This paper introduces GEDI, an electoral decision-making module for LLM-based multi-agent systems, demonstrating improved reasoning, robustness, and diversity through voting mechanisms, addressing current lack of diversity in collective decision-making.

Contribution

The paper proposes GEDI, a novel electoral CDM module using ordinal voting, enhancing diversity and robustness in LLM multi-agent decision-making frameworks.

Findings

Voting methods improve reasoning and robustness of LLMs.

Some CDM mechanisms work well with as few as three agents.

Voting-based methods are robust against single points of failure.

Abstract

Modern large language models (LLMs) have exhibited cooperative synergy on complex task-solving, and collective decision-making (CDM) is a pivotal component in LLM-based multi-agent collaboration frameworks. Our survey on 52 recent such systems uncovers a severe lack of diversity, with a heavy reliance on dictatorial and plurality voting for CDM. Through the lens of social choice theory, we scrutinize widely-adopted CDM methods and identify their limitations. To enrich current landscape of LLM-based CDM, we present GEDI, an electoral CDM module that incorporates various ordinal preferential voting mechanisms. Our empirical case study across three benchmarks shows that the integration of certain CDM methods can markedly improve the reasoning capabilities and robustness of some leading LLMs, all without requiring intricate system designs. Additionally, we find that some CDM mechanisms generate positive synergies even with as few as three agents. The voting-based methods also demonstrate robustness against single points of failure, as well as diversity in terms of hit-rate@k and subject-wise impacts.