Curriculum Guided Massive Multi Agent System Solving For Robust Long Horizon Tasks

TL;DR

This paper presents a hierarchical multi-agent system with a spatial curriculum and confidence-based training to improve long-horizon reasoning and robustness in complex tasks, evaluated on a Tower of Hanoi benchmark.

Contribution

Introduces a novel multi-agent architecture with a spatial curriculum and confidence measures, enhancing long-horizon reasoning and reliability in distributed systems.

Findings

Improved stability and reasoning in long-horizon tasks.

Reduced reliance on the oracle through confidence-based training.

Effective spatial curriculum accelerates agent mastery of complex tasks.

Abstract

Large Language Models and multi-agent systems have shown promise in decomposing complex tasks, yet they struggle with long-horizon reasoning tasks and escalating computation cost. This work introduces a hierarchical multi-agent architecture that distributes reasoning across a 64*64 grid of lightweight agents, supported by a selective oracle. A spatial curriculum progressively expands the operational region of the grid, ensuring that agents master easier central tasks before tackling harder peripheral ones. To improve reliability, the system integrates Negative Log-Likelihood as a measure of confidence, allowing the curriculum to prioritize regions where agents are both accurate and well calibrated. A Thompson Sampling curriculum manager adaptively chooses training zones based on competence and NLL-driven reward signals. We evaluate the approach on a spatially grounded Tower of Hanoi benchmark, which mirrors the long-horizon structure of many robotic manipulation and planning tasks. Results demonstrate improved stability, reduced oracle usage, and stronger long-range reasoning from distributed agent cooperation.