A Relearning Approach to Reinforcement Learning for Control of Smart Buildings

TL;DR

This paper proposes an incremental deep reinforcement learning approach for adaptive HVAC control in smart buildings, effectively handling non-stationary conditions to optimize energy use and maintain comfort.

Contribution

It introduces a relearning method for reinforcement learning that adapts to changing building and weather conditions without risking safety.

Findings

Relearning controller maintains energy efficiency over time.

Static controller performance deteriorates with non-stationarity.

Relearning approach ensures comfort and energy savings.

Abstract

This paper demonstrates that continual relearning of control policies using incremental deep reinforcement learning (RL) can improve policy learning for non-stationary processes. We demonstrate this approach for a data-driven 'smart building environment' that we use as a test-bed for developing HVAC controllers for reducing energy consumption of large buildings on our university campus. The non-stationarity in building operations and weather patterns makes it imperative to develop control strategies that are adaptive to changing conditions. On-policy RL algorithms, such as Proximal Policy Optimization (PPO) represent an approach for addressing this non-stationarity, but exploration on the actual system is not an option for safety-critical systems. As an alternative, we develop an incremental RL technique that simultaneously reduces building energy consumption without sacrificing overall comfort. We compare the performance of our incremental RL controller to that of a static RL controller that does not implement the relearning function. The performance of the static controller diminishes significantly over time, but the relearning controller adjusts to changing conditions while ensuring comfort and optimal energy performance.