A Safe and Data-efficient Model-based Reinforcement Learning System for HVAC Control

TL;DR

This paper introduces CLUE, a data-efficient, uncertainty-aware model-based reinforcement learning system for HVAC control that significantly reduces training data needs while maintaining comfort and energy efficiency.

Contribution

The paper presents a novel MBRL system using Gaussian Processes with meta-kernel learning for efficient building dynamics modeling in HVAC control.

Findings

Reduces training data from hundreds of days to seven.

Decreases comfort violations by 12.07% on average.

Maintains robust control performance with less data.

Abstract

Model-Based Reinforcement Learning (MBRL) has been widely studied for Heating, Ventilation, and Air Conditioning (HVAC) control in buildings. One of the critical challenges is the large amount of data required to effectively train neural networks for modeling building dynamics. This paper presents CLUE, an MBRL system for HVAC control in buildings. CLUE optimizes HVAC operations by integrating a Gaussian Process (GP) model to model building dynamics with uncertainty awareness. CLUE utilizes GP to predict state transitions as Gaussian distributions, effectively capturing prediction uncertainty and enhancing decision-making under sparse data conditions. Our approach employs a meta-kernel learning technique to efficiently set GP kernel hyperparameters using domain knowledge from diverse buildings. This drastically reduces the data requirements typically associated with GP models in HVAC applications. Additionally, CLUE incorporates these uncertainty estimates into a Model Predictive Path Integral (MPPI) algorithm, enabling the selection of safe, energy-efficient control actions. This uncertainty-aware control strategy evaluates and selects action trajectories based on their predicted impact on energy consumption and human comfort, optimizing operations even under uncertain conditions. Extensive simulations in a five-zone office building demonstrate that CLUE reduces the required training data from hundreds of days to just seven while maintaining robust control performance. It reduces comfort violations by an average of 12.07% compared to existing MBRL methods, without compromising on energy efficiency.