Task-agnostic Decision Transformer for Multi-type Agent Control with Federated Split Training

TL;DR

This paper introduces FSDT, a federated split decision transformer framework that enables privacy-preserving, efficient, and effective multi-agent decision-making using distributed offline reinforcement learning data.

Contribution

The paper presents a novel federated split decision transformer framework tailored for personalized multi-agent control, addressing aggregation challenges and reducing communication overhead.

Findings

FSDT outperforms traditional methods on D4RL benchmark.

Significant reduction in communication and computational costs.

Effective in privacy-preserving collaborative offline reinforcement learning.

Abstract

With the rapid advancements in artificial intelligence, the development of knowledgeable and personalized agents has become increasingly prevalent. However, the inherent variability in state variables and action spaces among personalized agents poses significant aggregation challenges for traditional federated learning algorithms. To tackle these challenges, we introduce the Federated Split Decision Transformer (FSDT), an innovative framework designed explicitly for AI agent decision tasks. The FSDT framework excels at navigating the intricacies of personalized agents by harnessing distributed data for training while preserving data privacy. It employs a two-stage training process, with local embedding and prediction models on client agents and a global transformer decoder model on the server. Our comprehensive evaluation using the benchmark D4RL dataset highlights the superior performance of our algorithm in federated split learning for personalized agents, coupled with significant reductions in communication and computational overhead compared to traditional centralized training approaches. The FSDT framework demonstrates strong potential for enabling efficient and privacy-preserving collaborative learning in applications such as autonomous driving decision systems. Our findings underscore the efficacy of the FSDT framework in effectively leveraging distributed offline reinforcement learning data to enable powerful multi-type agent decision systems.