Policy Search, Retrieval, and Composition via Task Similarity in Collaborative Agentic Systems

TL;DR

This paper introduces MOSAIC, a novel algorithm enabling agentic systems to efficiently share, select, and compose policies based on task similarity, significantly enhancing learning speed and performance in multi-agent environments.

Contribution

The study presents MOSAIC, a new method for policy sharing and composition that improves collective learning by leveraging task similarity and modular neural representations.

Findings

MOSAIC outperforms isolated and global sharing methods in speed and performance.

Selective reuse reduces task interference and enhances learning efficiency.

Shared knowledge accelerates learning of complex tasks through self-organization.

Abstract

Agentic AI aims to create systems that set their own goals, adapt proactively to change, and refine behavior through continuous experience. Recent advances suggest that, when facing multiple and unforeseen tasks, agents could benefit from sharing machine-learned knowledge and reusing policies that have already been fully or partially learned by other agents. However, how to query, select, and retrieve policies from a pool of agents, and how to integrate such policies remains a largely unexplored area. This study explores how an agent decides what knowledge to select, from whom, and when and how to integrate it in its own policy in order to accelerate its own learning. The proposed algorithm, \emph{Modular Sharing and Composition in Collective Learning} (MOSAIC), improves learning in agentic collectives by combining (1) knowledge selection using performance signals and cosine similarity on Wasserstein task embeddings, (2) modular and transferable neural representations via masks, and (3) policy integration, composition and fine-tuning. MOSAIC outperforms isolated learners and global sharing approaches in both learning speed and overall performance, and in some cases solves tasks that isolated agents cannot. The results also demonstrate that selective, goal-driven reuse leads to less susceptibility to task interference. We also observe the emergence of self-organization, where agents solving simpler tasks accelerate the learning of harder ones through shared knowledge.