Federated Deep Reinforcement Learning

TL;DR

This paper introduces FedRL, a federated deep reinforcement learning framework that enables multiple agents to collaboratively learn high-quality policies while preserving data and model privacy using Gaussian differentials.

Contribution

It presents a novel federated learning approach for deep reinforcement learning that protects privacy through Gaussian differentials during model updates.

Findings

FedRL outperforms baseline methods in Grid-world and Text2Action domains.

The framework effectively preserves privacy without sacrificing learning quality.

Experimental results demonstrate the viability of privacy-aware federated reinforcement learning.

Abstract

In deep reinforcement learning, building policies of high-quality is challenging when the feature space of states is small and the training data is limited. Despite the success of previous transfer learning approaches in deep reinforcement learning, directly transferring data or models from an agent to another agent is often not allowed due to the privacy of data and/or models in many privacy-aware applications. In this paper, we propose a novel deep reinforcement learning framework to federatively build models of high-quality for agents with consideration of their privacies, namely Federated deep Reinforcement Learning (FedRL). To protect the privacy of data and models, we exploit Gausian differentials on the information shared with each other when updating their local models. In the experiment, we evaluate our FedRL framework in two diverse domains, Grid-world and Text2Action domains, by comparing to various baselines.