Optimal Operation of a Hydrogen-based Building Multi-Energy System Based on Deep Reinforcement Learning

TL;DR

This paper presents a deep reinforcement learning approach for optimally operating hydrogen-based multi-energy systems, incorporating building thermal dynamics to reduce costs without needing explicit thermal models.

Contribution

It introduces a novel multi-agent reinforcement learning algorithm that handles uncertainties and complex constraints in HMES operation, considering building thermal dynamics.

Findings

The proposed algorithm effectively reduces operational costs.

It does not require prior knowledge of uncertain parameters.

Simulation results validate the approach's effectiveness.

Abstract

Since hydrogen has many advantages (e.g., free pollution, extensive sources, convenient storage and transportation), hydrogen-based multi-energy systems (HMESs) have received wide attention. However, existing works on the optimal operation of HMESs neglect building thermal dynamics, which means that the flexibility of building thermal loads can not be utilized for reducing system operation cost. In this paper, we investigate an optimal operation problem of an HMES with the consideration of building thermal dynamics. Specifically, we first formulate an expected operational cost minimization problem related to an HMES. Due to the existence of uncertain parameters, inexplicit building thermal dynamics models, temporally coupled operational constraints related to three kinds of energy storage systems and indoor temperatures, as well as the coupling between electric energy subsystems and thermal energy subsystems, it is challenging to solve the formulated problem. To overcome the challenge, we reformulate the problem as a Markov game and propose an energy management algorithm to solve it based on multi-agent discrete actor-critic with rules (MADACR). Note that the proposed algorithm does not require any prior knowledge of uncertain parameters, parameter prediction, and explicit building thermal dynamics model. Simulation results based on real-world traces show the effectiveness of the proposed algorithm.