Multi-Agent Debate Strategies to Enhance Requirements Engineering with Large Language Models

TL;DR

This paper explores the use of multi-agent debate strategies among large language models to improve the accuracy and robustness of requirements engineering tasks, moving beyond single-pass methods.

Contribution

It introduces a taxonomy of MAD strategies and demonstrates the feasibility of applying multi-agent debates to RE classification tasks.

Findings

Identified key characteristics of MAD strategies

Developed a taxonomy of MAD attributes

Preliminary evaluation shows MAD can enhance RE classification

Abstract

Context: Large Language Model (LLM) agents are becoming widely used for various Requirements Engineering (RE) tasks. Research on improving their accuracy mainly focuses on prompt engineering, model fine-tuning, and retrieval augmented generation. However, these methods often treat models as isolated black boxes - relying on single-pass outputs without iterative refinement or collaboration, limiting robustness and adaptability. Objective: We propose that, just as human debates enhance accuracy and reduce bias in RE tasks by incorporating diverse perspectives, different LLM agents debating and collaborating may achieve similar improvements. Our goal is to investigate whether Multi-Agent Debate (MAD) strategies can enhance RE performance. Method: We conducted a systematic study of existing MAD strategies across various domains to identify their key characteristics. To assess their applicability in RE, we implemented and tested a preliminary MAD-based framework for RE classification. Results: Our study identified and categorized several MAD strategies, leading to a taxonomy outlining their core attributes. Our preliminary evaluation demonstrated the feasibility of applying MAD to RE classification. Conclusions: MAD presents a promising approach for improving LLM accuracy in RE tasks. This study provides a foundational understanding of MAD strategies, offering insights for future research and refinements in RE applications.