An Interactive Multi-Agent System for Evaluation of New Product Concepts

TL;DR

This paper introduces an automated multi-agent system leveraging large language models to evaluate new product concepts objectively, efficiently, and with expert-level accuracy, addressing limitations of traditional subjective methods.

Contribution

It presents a novel multi-agent system using LLMs with retrieval-augmented generation for structured product concept evaluation based on technical and market feasibility.

Findings

System rankings aligned with industry experts

Demonstrated efficiency over traditional methods

Validated approach through a real-world case study

Abstract

Product concept evaluation is a critical stage that determines strategic resource allocation and project success in enterprises. However, traditional expert-led approaches face limitations such as subjective bias and high time and cost requirements. To support this process, this study proposes an automated approach utilizing a large language model (LLM)-based multi-agent system (MAS). Through a systematic analysis of previous research on product development and team collaboration, this study established two primary evaluation dimensions, namely technical feasibility and market feasibility. The proposed system consists of a team of eight virtual agents representing specialized domains such as R&D and marketing. These agents use retrieval-augmented generation (RAG) and real-time search tools to gather objective evidence and validate concepts through structured deliberations based on the established criteria. The agents were further fine-tuned using professional product review data to enhance their judgment accuracy. A case study involving professional display monitor concepts demonstrated that the system's evaluation rankings were consistent with those of senior industry experts. These results confirm the usability of the proposed multi-agent-based evaluation approach for supporting product development decisions.