Facilitating Trustworthy Human-Agent Collaboration in LLM-based Multi-Agent System oriented Software Engineering

TL;DR

This paper proposes a RACI-based framework to enhance trustworthy collaboration between humans and LLM-driven multi-agent systems in software engineering, addressing accountability and risk mitigation.

Contribution

It introduces a novel RACI-based framework with implementation guidelines to improve trustworthiness and task allocation in LLM-based multi-agent systems for software engineering.

Findings

Framework facilitates efficient human-agent collaboration

Ensures accountability and mitigates risks

Aligns with Trustworthy AI guidelines

Abstract

Multi-agent autonomous systems (MAS) are better at addressing challenges that spans across multiple domains than singular autonomous agents. This holds true within the field of software engineering (SE) as well. The state-of-the-art research on MAS within SE focuses on integrating LLMs at the core of autonomous agents to create LLM-based multi-agent autonomous (LMA) systems. However, the introduction of LMA systems into SE brings a plethora of challenges. One of the major challenges is the strategic allocation of tasks between humans and the LMA system in a trustworthy manner. To address this challenge, a RACI-based framework is proposed in this work in progress article, along with implementation guidelines and an example implementation of the framework. The proposed framework can facilitate efficient collaboration, ensure accountability, and mitigate potential risks associated with LLM-driven automation while aligning with the Trustworthy AI guidelines. The future steps for this work delineating the planned empirical validation method are also presented.