Distributed Multi-Agent Deep Reinforcement Learning Framework for Whole-building HVAC Control

TL;DR

This paper introduces a distributed multi-agent deep reinforcement learning framework that significantly reduces HVAC energy consumption in commercial buildings while maintaining occupant comfort, using simulation-based training.

Contribution

It presents a novel scalable DRL framework that models complex thermal dynamics and optimizes HVAC control for energy efficiency and comfort in buildings.

Findings

Achieves over 75% energy savings in simulations

Effectively balances energy costs and thermal comfort

Framework scalable to heterogeneous computing resources

Abstract

It is estimated that about 40%-50% of total electricity consumption in commercial buildings can be attributed to Heating, Ventilation, and Air Conditioning (HVAC) systems. Minimizing the energy cost while considering the thermal comfort of the occupants is very challenging due to unknown and complex relationships between various HVAC controls and thermal dynamics inside a building. To this end, we present a multi-agent, distributed deep reinforcement learning (DRL) framework based on Energy Plus simulation environment for optimizing HVAC in commercial buildings. This framework learns the complex thermal dynamics in the building and takes advantage of the differential effect of cooling and heating systems in the building to reduce energy costs, while maintaining the thermal comfort of the occupants. With adaptive penalty, the RL algorithm can be prioritized for energy savings or maintaining thermal comfort. Using DRL, we achieve more than 75\% savings in energy consumption. The distributed DRL framework can be scaled to multiple GPUs and CPUs of heterogeneous types.