Data-Driven Policy Mapping for Safe RL-based Energy Management Systems

TL;DR

This paper introduces a scalable, safe, and adaptive RL-based building energy management system that uses clustering, forecasting, and action masking to optimize costs and ensure safety across diverse buildings.

Contribution

The paper presents a novel three-step RL framework combining clustering, forecasting, and domain-informed action masking for scalable and safe energy management in buildings.

Findings

Reduces operating costs by up to 15% in real-world tests

Maintains stable environmental performance across building types

Adapts to tariff changes without retraining

Abstract

Increasing global energy demand and renewable integration complexity have placed buildings at the center of sustainable energy management. We present a three-step reinforcement learning(RL)-based Building Energy Management System (BEMS) that combines clustering, forecasting, and constrained policy learning to address scalability, adaptability, and safety challenges. First, we cluster non-shiftable load profiles to identify common consumption patterns, enabling policy generalization and transfer without retraining for each new building. Next, we integrate an LSTM based forecasting module to anticipate future states, improving the RL agents' responsiveness to dynamic conditions. Lastly, domain-informed action masking ensures safe exploration and operation, preventing harmful decisions. Evaluated on real-world data, our approach reduces operating costs by up to 15% for certain building types, maintains stable environmental performance, and quickly classifies and optimizes new buildings with limited data. It also adapts to stochastic tariff changes without retraining. Overall, this framework delivers scalable, robust, and cost-effective building energy management.