Personalized Federated Hypernetworks for Privacy Preservation in Multi-Task Reinforcement Learning

TL;DR

This paper introduces Personalized Federated Hypernetworks (PFH) for privacy-preserving multi-task reinforcement learning, demonstrating their effectiveness in energy demand response applications and few-shot transfer scenarios.

Contribution

It is the first to apply PFH to reinforcement learning, combining federated learning with hypernetworks for privacy and efficiency in multi-task RL settings.

Findings

PFH improves learning efficiency in multi-task RL.

PFH enables privacy preservation across microgrids.

Significant initial gains in few-shot transfer learning.

Abstract

Multi-Agent Reinforcement Learning currently focuses on implementations where all data and training can be centralized to one machine. But what if local agents are split across multiple tasks, and need to keep data private between each? We develop the first application of Personalized Federated Hypernetworks (PFH) to Reinforcement Learning (RL). We then present a novel application of PFH to few-shot transfer, and demonstrate significant initial increases in learning. PFH has never been demonstrated beyond supervised learning benchmarks, so we apply PFH to an important domain: RL price-setting for energy demand response. We consider a general case across where agents are split across multiple microgrids, wherein energy consumption data must be kept private within each microgrid. Together, our work explores how the fields of personalized federated learning and RL can come together to make learning efficient across multiple tasks while keeping data secure.