GoNoGo: An Efficient LLM-based Multi-Agent System for Streamlining Automotive Software Release Decision-Making

TL;DR

GoNoGo is an LLM-based multi-agent system that automates and streamlines automotive software release decisions, reducing manual effort and improving reliability in risk-sensitive vehicle systems.

Contribution

The paper introduces GoNoGo, a domain-specific LLM agent system tailored for automotive software deployment, addressing practical industrial constraints and reducing prompt engineering.

Findings

Achieves 100% success on tasks up to Level 2 difficulty with 3-shot examples.

Effectively automates decision-making for simpler tasks, reducing manual intervention.

Maintains high performance on complex tasks, supporting industrial deployment.

Abstract

Traditional methods for making software deployment decisions in the automotive industry typically rely on manual analysis of tabular software test data. These methods often lead to higher costs and delays in the software release cycle due to their labor-intensive nature. Large Language Models (LLMs) present a promising solution to these challenges. However, their application generally demands multiple rounds of human-driven prompt engineering, which limits their practical deployment, particularly for industrial end-users who need reliable and efficient results. In this paper, we propose GoNoGo, an LLM agent system designed to streamline automotive software deployment while meeting both functional requirements and practical industrial constraints. Unlike previous systems, GoNoGo is specifically tailored to address domain-specific and risk-sensitive systems. We evaluate GoNoGo's performance across different task difficulties using zero-shot and few-shot examples taken from industrial practice. Our results show that GoNoGo achieves a 100% success rate for tasks up to Level 2 difficulty with 3-shot examples, and maintains high performance even for more complex tasks. We find that GoNoGo effectively automates decision-making for simpler tasks, significantly reducing the need for manual intervention. In summary, GoNoGo represents an efficient and user-friendly LLM-based solution currently employed in our industrial partner's company to assist with software release decision-making, supporting more informed and timely decisions in the release process for risk-sensitive vehicle systems.