Safe Deep Reinforcement Learning for Building Heating Control and Demand-side Flexibility

TL;DR

This paper introduces a safe deep reinforcement learning framework for building heating control that ensures demand-side flexibility, occupant comfort, and energy efficiency, with real-time safety guarantees and significant cost savings.

Contribution

It develops a real-time adaptive safety filter integrated with deep reinforcement learning to ensure safety and compliance in demand-side flexibility for building heating systems.

Findings

Achieves up to 50% energy and cost savings compared to rule-based control.

Outperforms standalone deep reinforcement learning controllers in efficiency.

Maintains occupant comfort with only slight temperature violations.

Abstract

Buildings account for approximately 40% of global energy consumption, and with the growing share of intermittent renewable energy sources, enabling demand-side flexibility, particularly in heating, ventilation and air conditioning systems, is essential for grid stability and energy efficiency. This paper presents a safe deep reinforcement learning-based control framework to optimize building space heating while enabling demand-side flexibility provision for power system operators. A deep deterministic policy gradient algorithm is used as the core deep reinforcement learning method, enabling the controller to learn an optimal heating strategy through interaction with the building thermal model while maintaining occupant comfort, minimizing energy cost, and providing flexibility. To address safety concerns with reinforcement learning, particularly regarding compliance with flexibility requests, we propose a real-time adaptive safety-filter to ensure that the system operates within predefined constraints during demand-side flexibility provision. The proposed real-time adaptive safety filter guarantees full compliance with flexibility requests from system operators and improves energy and cost efficiency -- achieving up to 50% savings compared to a rule-based controller -- while outperforming a standalone deep reinforcement learning-based controller in energy and cost metrics, with only a slight increase in comfort temperature violations.