Near-optimal Deep Reinforcement Learning Policies from Data for Zone Temperature Control

TL;DR

This paper demonstrates that Deep Reinforcement Learning agents can achieve near-optimal control performance in building zone temperature management by using Physically Consistent Neural Networks as simulation environments, outperforming traditional controllers.

Contribution

The study introduces the use of PCNNs for evaluating DRL policies, showing they can reach near-optimal performance without complex physics modeling.

Findings

DRL agents outperform rule-based controllers

DRL agents achieve near-optimal control performance

PCNNs effectively evaluate control policies

Abstract

Replacing poorly performing existing controllers with smarter solutions will decrease the energy intensity of the building sector. Recently, controllers based on Deep Reinforcement Learning (DRL) have been shown to be more effective than conventional baselines. However, since the optimal solution is usually unknown, it is still unclear if DRL agents are attaining near-optimal performance in general or if there is still a large gap to bridge. In this paper, we investigate the performance of DRL agents compared to the theoretically optimal solution. To that end, we leverage Physically Consistent Neural Networks (PCNNs) as simulation environments, for which optimal control inputs are easy to compute. Furthermore, PCNNs solely rely on data to be trained, avoiding the difficult physics-based modeling phase, while retaining physical consistency. Our results hint that DRL agents not only clearly outperform conventional rule-based controllers, they furthermore attain near-optimal performance.