Agentic Neural Networks: Self-Evolving Multi-Agent Systems via Textual Backpropagation

TL;DR

The paper introduces Agentic Neural Networks, a novel framework that enables self-evolving multi-agent systems using a layered neural architecture and a two-phase optimization process inspired by backpropagation, improving adaptability and performance.

Contribution

It presents a neuro-symbolic, layered neural network framework for multi-agent collaboration that self-evolves roles and prompts through a novel forward-backward optimization strategy.

Findings

Outperforms existing multi-agent baselines on four benchmark datasets.

Enables agents to self-evolve roles, prompts, and coordination.

Achieves notable gains in accuracy and adaptability.

Abstract

Leveraging multiple Large Language Models(LLMs) has proven effective for addressing complex, high-dimensional tasks, but current approaches often rely on static, manually engineered multi-agent configurations. To overcome these constraints, we present the Agentic Neural Network(ANN), a framework that conceptualizes multi-agent collaboration as a layered neural network architecture. In this design, each agent operates as a node, and each layer forms a cooperative "team" focused on a specific subtask. Agentic Neural Network follows a two-phase optimization strategy: (1) Forward Phase-Drawing inspiration from neural network forward passes, tasks are dynamically decomposed into subtasks, and cooperative agent teams with suitable aggregation methods are constructed layer by layer. (2) Backward Phase-Mirroring backpropagation, we refine both global and local collaboration through iterative feedback, allowing agents to self-evolve their roles, prompts, and coordination. This neuro-symbolic approach enables ANN to create new or specialized agent teams post-training, delivering notable gains in accuracy and adaptability. Across four benchmark datasets, ANN surpasses leading multi-agent baselines under the same configurations, showing consistent performance improvements. Our findings indicate that ANN provides a scalable, data-driven framework for multi-agent systems, combining the collaborative capabilities of LLMs with the efficiency and flexibility of neural network principles. We plan to open-source the entire framework.