Think-on-Process: Dynamic Process Generation for Collaborative Development of Multi-Agent System

TL;DR

This paper introduces ToP, a dynamic process generation framework that uses experiential knowledge and process mining to enable LLM-guided multi-agent software development adaptable to real-time changes.

Contribution

The paper presents a novel framework that dynamically generates and refines software development processes using LLMs, process mining, and heuristic filtering, improving flexibility over fixed process models.

Findings

Enhances GPT-3.5 and GPT-4's process generation capabilities.

Significantly improves adaptability for five software development task categories.

Uses process mining to derive and refine process models.

Abstract

Software development is a collaborative endeavor that requires individuals from different departments to work together in order to collectively develop a high-quality software system. In this context, people have begun to explore a method that leverages multi-agent systems based on LLMs to carry out software development. However, existing research tends to rigidly fix the software development process in a framework in code form, thus failing to dynamically adjust the software development process in real-time to meet the more flexible and variable software environment. In this paper, we propose a dynamic process generation framework, named ToP (Think-on-Process). The core idea of ToP is to leverage experiential knowledge (i.e., process models) to guide LLMs in generating software development processes (i.e., instances). These instances will guide multi-agent in software development and employ a compiler to provide feedback on the development outcomes. Subsequently, we utilize heuristic algorithms to filter the instances and apply process mining algorithms to derive process model. Finally, the process model will be converted into text, formatted as prompts, to enhance the ability of LLMs to generate other instances. Experiments demonstrate that our framework ToP significantly enhances the dynamic process generation capability of the GPT-3.5 and GPT-4 for five categories of software development tasks.