Federated In-Context LLM Agent Learning

TL;DR

This paper introduces FICAL, a privacy-preserving federated learning framework that leverages in-context learning and knowledge compendiums to efficiently train diverse LLM agents with reduced communication costs.

Contribution

The paper proposes a novel FICAL algorithm that uses knowledge compendiums instead of model parameters and incorporates RAG-based tool learning, advancing federated LLM training with privacy and efficiency.

Findings

FICAL achieves competitive performance with state-of-the-art methods.

Significantly reduces communication costs by over 3.33×10^5 times.

Demonstrates effective tool learning and knowledge sharing among LLM agents.

Abstract

Large Language Models (LLMs) have revolutionized intelligent services by enabling logical reasoning, tool use, and interaction with external systems as agents. The advancement of LLMs is frequently hindered by the scarcity of high-quality data, much of which is inherently sensitive. Federated learning (FL) offers a potential solution by facilitating the collaborative training of distributed LLMs while safeguarding private data. However, FL frameworks face significant bandwidth and computational demands, along with challenges from heterogeneous data distributions. The emerging in-context learning capability of LLMs offers a promising approach by aggregating natural language rather than bulky model parameters. Yet, this method risks privacy leakage, as it necessitates the collection and presentation of data samples from various clients during aggregation. In this paper, we propose a novel privacy-preserving Federated In-Context LLM Agent Learning (FICAL) algorithm, which to our best knowledge for the first work unleashes the power of in-context learning to train diverse LLM agents through FL. In our design, knowledge compendiums generated by a novel LLM-enhanced Knowledge Compendiums Generation (KCG) module are transmitted between clients and the server instead of model parameters in previous FL methods. Apart from that, an incredible Retrieval Augmented Generation (RAG) based Tool Learning and Utilizing (TLU) module is designed and we incorporate the aggregated global knowledge compendium as a teacher to teach LLM agents the usage of tools. We conducted extensive experiments and the results show that FICAL has competitive performance compared to other SOTA baselines with a significant communication cost decrease of $\mathbf{3.33\times10^5}$ times.