Does Explicit Prediction Matter in Deep Reinforcement Learning-Based Energy Management?

TL;DR

This paper compares energy management schemes with and without prediction in deep reinforcement learning, finding that end-to-end DRL without prediction outperforms prediction-based methods, challenging common assumptions.

Contribution

It demonstrates that end-to-end DRL without prediction modules can outperform traditional prediction-based schemes in energy management tasks.

Findings

End-to-end DRL without prediction is superior.

Prediction modules may be unnecessary or even harmful.

The study challenges the conventional use of prediction in DRL energy management.

Abstract

As a model-free optimization and decision-making method, deep reinforcement learning (DRL) has been widely applied to the filed of energy management in energy Internet. While, some DRL-based energy management schemes also incorporate the prediction module used by the traditional model-based methods, which seems to be unnecessary and even adverse. In this work, we implement the standard energy management scheme with prediction using supervised learning and DRL, and the counterpart without prediction using end-to-end DRL. Then, these two schemes are compared in the unified energy management framework. The simulation results demonstrate that the energy management scheme without prediction is superior over the scheme with prediction. This work intends to rectify the misuse of DRL methods in the field of energy management.