A New Error Temporal Difference Algorithm for Deep Reinforcement Learning in Microgrid Optimization

TL;DR

This paper introduces a novel error temporal difference (ETD) algorithm for deep reinforcement learning to better handle prediction uncertainties in microgrid energy management, leading to improved operational performance.

Contribution

The paper proposes a new ETD algorithm specifically designed to address uncertainty in DRL-based microgrid control, enhancing optimization results.

Findings

ETD improves DRL performance in microgrid management.

Simulations show better energy optimization with ETD.

The approach effectively manages prediction uncertainties.

Abstract

Predictive control approaches based on deep reinforcement learning (DRL) have gained significant attention in microgrid energy optimization. However, existing research often overlooks the issue of uncertainty stemming from imperfect prediction models, which can lead to suboptimal control strategies. This paper presents a new error temporal difference (ETD) algorithm for DRL to address the uncertainty in predictions,aiming to improve the performance of microgrid operations. First,a microgrid system integrated with renewable energy sources (RES) and energy storage systems (ESS), along with its Markov decision process (MDP), is modelled. Second, a predictive control approach based on a deep Q network (DQN) is presented, in which a weighted average algorithm and a new ETD algorithm are designed to quantify and address the prediction uncertainty, respectively. Finally, simulations on a realworld US dataset suggest that the developed ETD effectively improves the performance of DRL in optimizing microgrid operations.