Exploring Design of Multi-Agent LLM Dialogues for Research Ideation

TL;DR

This paper analyzes how different configurations of multi-agent LLM dialogues affect the quality of research ideas, providing guidelines for designing effective systems to enhance idea novelty and feasibility.

Contribution

It systematically evaluates the impact of agent roles, number, and diversity in multi-agent LLM dialogues for scientific ideation, offering practical design insights.

Findings

Increasing agent cohort size enriches idea diversity.

Deeper interactions improve idea quality.

Diverse critics enhance idea feasibility.

Abstract

Large language models (LLMs) are increasingly used to support creative tasks such as research idea generation. While recent work has shown that structured dialogues between LLMs can improve the novelty and feasibility of generated ideas, the optimal design of such interactions remains unclear. In this study, we conduct a comprehensive analysis of multi-agent LLM dialogues for scientific ideation. We compare different configurations of agent roles, number of agents, and dialogue depth to understand how these factors influence the novelty and feasibility of generated ideas. Our experimental setup includes settings where one agent generates ideas and another critiques them, enabling iterative improvement. Our results show that enlarging the agent cohort, deepening the interaction depth, and broadening agent persona heterogeneity each enrich the diversity of generated ideas. Moreover, specifically increasing critic-side diversity within the ideation-critique-revision loop further boosts the feasibility of the final proposals. Our findings offer practical guidelines for building effective multi-agent LLM systems for scientific ideation. Our code is available at https://github.com/g6000/MultiAgent-Research-Ideator.