On Decentralizing Federated Reinforcement Learning in Multi-Robot Scenarios

TL;DR

This paper proposes a decentralized federated reinforcement learning approach for multi-robot systems, utilizing mobile agents to avoid central server failures and bandwidth bottlenecks, demonstrated through simulations with Q-learning and SARSA.

Contribution

It introduces a mobile agent-based paradigm for decentralizing federated learning in multi-robot scenarios, enhancing robustness and scalability.

Findings

Decentralized FL is viable with mobile agents in robotics.

Experiments with Q-learning and SARSA show effective model aggregation.

Decentralized FL can be integrated with various learning algorithms.

Abstract

Federated Learning (FL) allows for collaboratively aggregating learned information across several computing devices and sharing the same amongst them, thereby tackling issues of privacy and the need of huge bandwidth. FL techniques generally use a central server or cloud for aggregating the models received from the devices. Such centralized FL techniques suffer from inherent problems such as failure of the central node and bottlenecks in channel bandwidth. When FL is used in conjunction with connected robots serving as devices, a failure of the central controlling entity can lead to a chaotic situation. This paper describes a mobile agent based paradigm to decentralize FL in multi-robot scenarios. Using Webots, a popular free open-source robot simulator, and Tartarus, a mobile agent platform, we present a methodology to decentralize federated learning in a set of connected robots. With Webots running on different connected computing systems, we show how mobile agents can perform the task of Decentralized Federated Reinforcement Learning (dFRL). Results obtained from experiments carried out using Q-learning and SARSA by aggregating their corresponding Q-tables, show the viability of using decentralized FL in the domain of robotics. Since the proposed work can be used in conjunction with other learning algorithms and also real robots, it can act as a vital tool for the study of decentralized FL using heterogeneous learning algorithms concurrently in multi-robot scenarios.