Active Learning for Communication Structure Optimization in LLM-Based Multi-Agent Systems

TL;DR

This paper introduces an ensemble-based, information-theoretic task selection framework to optimize communication structures in LLM-based multi-agent systems, improving performance and efficiency under limited training budgets.

Contribution

It proposes a novel task selection method that estimates task informativeness via distribution changes, enhancing communication-structure optimization in multi-agent systems.

Findings

Effective in both benign and adversarial settings.

Improves optimization stability with limited budgets.

Reduces token usage while maintaining performance.

Abstract

Optimizing the communication structure of large language model based multi-agent systems (LLM-MAS) has been shown to improve downstream performance and reduce token usage. Existing methods typically rely on randomly sampled training tasks. However, tasks may differ substantially in difficulty and domain, and thus they are not equally informative for updating communication structure, making optimization under limited training budgets often unstable and highly sensitive to the particular training set. To actively identify the most valuable tasks for communication-structure optimization, we propose an ensemble-based information-theoretic task selection framework. The proposed method estimates task informativeness by how much a candidate task changes the distribution over graph parameters, using ensemble Kalman inversion as an efficient and derivative-free approximation of the corresponding Bayesian update. The resulting estimator is especially suitable for black-box and noisy multi-agent systems. To enhance scalability, we construct a compact candidate pool through embedding-based representative selection and combine the informative selection with surrogate modeling and batch Thompson sampling. We validate our method in both benign settings and settings with agent attacks, demonstrating its effectiveness for communication-structure optimization under constrained computational budgets.