A Federated DRL Approach for Smart Micro-Grid Energy Control with Distributed Energy Resources

TL;DR

This paper introduces a federated deep reinforcement learning framework for managing energy in smart micro-grids with distributed energy resources, improving efficiency, reducing emissions, and protecting privacy.

Contribution

It presents a novel hierarchical federated DRL architecture for decentralized energy management in smart buildings with renewable resources.

Findings

The framework reduces costs and CO2 emissions.

It accelerates learning through federated knowledge sharing.

Performance is validated across various environmental conditions.

Abstract

The prevalence of the Internet of things (IoT) and smart meters devices in smart grids is providing key support for measuring and analyzing the power consumption patterns. This approach enables end-user to play the role of prosumers in the market and subsequently contributes to diminish the carbon footprint and the burden on utility grids. The coordination of trading surpluses of energy that is generated by house renewable energy resources (RERs) and the supply of shortages by external networks (main grid) is a necessity. This paper proposes a hierarchical architecture to manage energy in multiple smart buildings leveraging federated deep reinforcement learning (FDRL) with dynamic load in a distributed manner. Within the context of the developed FDRL-based framework, each agent that is hosted in local building energy management systems (BEMS) trains a local deep reinforcement learning (DRL) model and shares its experience in the form of model hyperparameters to the federation layer in the energy management system (EMS). Simulation studies are conducted using one EMS and up to twenty smart houses that are equipped with photovoltaic (PV) systems and batteries. This iterative training approach enables the proposed discretized soft actor-critic (SAC) agents to aggregate the collected knowledge to expedite the overall learning procedure and reduce costs and CO2 emissions, while the federation approach can mitigate privacy breaches. The numerical results confirm the performance of the proposed framework under different daytime periods, loads, and temperatures.