Optimal Management of Grid-Interactive Efficient Buildings via Safe Reinforcement Learning

TL;DR

This paper introduces a physics-inspired safe reinforcement learning method for managing grid-interactive efficient buildings, ensuring strict safety constraints while optimizing energy costs and comfort.

Contribution

It develops a novel safe RL approach with enforced safety guarantees using hard steady-state rules, addressing limitations of existing methods.

Findings

Achieves strict safety constraints in GEB control.

Reduces energy costs while maintaining customer comfort.

Demonstrates effectiveness through simulations on HVAC, PV, and storage systems.

Abstract

Reinforcement learning (RL)-based methods have achieved significant success in managing grid-interactive efficient buildings (GEBs). However, RL does not carry intrinsic guarantees of constraint satisfaction, which may lead to severe safety consequences. Besides, in GEB control applications, most existing safe RL approaches rely only on the regularisation parameters in neural networks or penalty of rewards, which often encounter challenges with parameter tuning and lead to catastrophic constraint violations. To provide enforced safety guarantees in controlling GEBs, this paper designs a physics-inspired safe RL method whose decision-making is enhanced through safe interaction with the environment. Different energy resources in GEBs are optimally managed to minimize energy costs and maximize customer comfort. The proposed approach can achieve strict constraint guarantees based on prior knowledge of a set of developed hard steady-state rules. Simulations on the optimal management of GEBs, including heating, ventilation, and air conditioning (HVAC), solar photovoltaics, and energy storage systems, demonstrate the effectiveness of the proposed approach.