Exploring Communication Strategies for Collaborative LLM Agents in Mathematical Problem-Solving

TL;DR

This paper systematically evaluates four communication strategies among LLM agents in collaborative mathematical problem-solving, demonstrating that peer-to-peer collaboration yields the highest accuracy and emphasizing the importance of dialogue acts.

Contribution

It introduces a comparative analysis of communication modes in dual-agent LLM environments for math problem-solving, highlighting the impact of dialogue strategies on performance.

Findings

Dual-agent setups outperform single agents.

Peer-to-peer collaboration achieves the highest accuracy.

Dialogue acts significantly influence problem-solving effectiveness.

Abstract

Large Language Model (LLM) agents are increasingly utilized in AI-aided education to support tutoring and learning. Effective communication strategies among LLM agents improve collaborative problem-solving efficiency and facilitate cost-effective adoption in education. However, little research has systematically evaluated the impact of different communication strategies on agents' problem-solving. Our study examines four communication modes, \textit{teacher-student interaction}, \textit{peer-to-peer collaboration}, \textit{reciprocal peer teaching}, and \textit{critical debate}, in a dual-agent, chat-based mathematical problem-solving environment using the OpenAI GPT-4o model. Evaluated on the MATH dataset, our results show that dual-agent setups outperform single agents, with \textit{peer-to-peer collaboration} achieving the highest accuracy. Dialogue acts like statements, acknowledgment, and hints play a key role in collaborative problem-solving. While multi-agent frameworks enhance computational tasks, effective communication strategies are essential for tackling complex problems in AI education.