Multi-agent Bayesian Deep Reinforcement Learning for Microgrid Energy Management under Communication Failures

TL;DR

This paper introduces a multi-agent Bayesian deep reinforcement learning approach for microgrid energy management that remains effective despite communication failures, improving decision-making robustness in IoT-enabled smart grids.

Contribution

It develops a novel BA-DRL framework incorporating belief updates and a belief-based equilibrium for resilient multi-agent coordination under communication uncertainties.

Findings

BA-DRL outperforms Nash-DQN and ADMM in reward under communication failures.

The method is robust to power supply and communication uncertainties.

Simulation results demonstrate improved energy management performance.

Abstract

Microgrids (MGs) are important players for the future transactive energy systems where a number of intelligent Internet of Things (IoT) devices interact for energy management in the smart grid. Although there have been many works on MG energy management, most studies assume a perfect communication environment, where communication failures are not considered. In this paper, we consider the MG as a multi-agent environment with IoT devices in which AI agents exchange information with their peers for collaboration. However, the collaboration information may be lost due to communication failures or packet loss. Such events may affect the operation of the whole MG. To this end, we propose a multi-agent Bayesian deep reinforcement learning (BA-DRL) method for MG energy management under communication failures. We first define a multi-agent partially observable Markov decision process (MA-POMDP) to describe agents under communication failures, in which each agent can update its beliefs on the actions of its peers. Then, we apply a double deep Q-learning (DDQN) architecture for Q-value estimation in BA-DRL, and propose a belief-based correlated equilibrium for the joint-action selection of multi-agent BA-DRL. Finally, the simulation results show that BA-DRL is robust to both power supply uncertainty and communication failure uncertainty. BA-DRL has 4.1% and 10.3% higher reward than Nash Deep Q-learning (Nash-DQN) and alternating direction method of multipliers (ADMM) respectively under 1% communication failure probability.