AI based Multiagent Approach for Requirements Elicitation and Analysis

TL;DR

This paper explores using a multi-agent AI system with various large language models to automate and improve requirements analysis in software engineering, demonstrating effectiveness through experiments on real projects.

Contribution

It introduces a multi-agent system deploying multiple LLMs for requirements elicitation, analysis, and prioritization, showcasing their comparative performance and practical benefits.

Findings

GPT-3.5 excels with complex user stories

Mixtral-8B provides fastest responses

LLMs improve RE efficiency and quality

Abstract

Requirements Engineering (RE) plays a pivotal role in software development, encompassing tasks such as requirements elicitation, analysis, specification, and change management. Despite its critical importance, RE faces challenges including communication complexities, early-stage uncertainties, and accurate resource estimation. This study empirically investigates the effectiveness of utilizing Large Language Models (LLMs) to automate requirements analysis tasks. We implemented a multi-agent system that deploys AI models as agents to generate user stories from initial requirements, assess and improve their quality, and prioritize them using a selected technique. In our implementation, we deployed four models, namely GPT-3.5, GPT-4 Omni, LLaMA3-70, and Mixtral-8B, and conducted experiments to analyze requirements on four real-world projects. We evaluated the results by analyzing the semantic similarity and API performance of different models, as well as their effectiveness and efficiency in requirements analysis, gathering users' feedback on their experiences. Preliminary results indicate notable variations in task completion among the models. Mixtral-8B provided the quickest responses, while GPT-3.5 performed exceptionally well when processing complex user stories with a higher similarity score, demonstrating its capability in deriving accurate user stories from project descriptions. Feedback and suggestions from the four project members further corroborate the effectiveness of LLMs in improving and streamlining RE phases.