Simulation Studies on Deep Reinforcement Learning for Building Control with Human Interaction

TL;DR

This paper explores the use of deep reinforcement learning, specifically DDPG, for building climate control with occupant interaction, demonstrating its potential to handle complex, uncertain, and partially observable environments in simulation.

Contribution

It applies DDPG to building control problems with occupant interaction, addressing partial observability and uncertainties, and evaluates its performance through simulation studies.

Findings

DDPG achieves reasonable control performance in simulations.

The approach handles high-dimensional, stochastic, and partially observable states.

Simulation results show computational feasibility of the method.

Abstract

The building sector consumes the largest energy in the world, and there have been considerable research interests in energy consumption and comfort management of buildings. Inspired by recent advances in reinforcement learning (RL), this paper aims at assessing the potential of RL in building climate control problems with occupant interaction. We apply a recent RL approach, called DDPG (deep deterministic policy gradient), for the continuous building control tasks and assess its performance with simulation studies in terms of its ability to handle (a) the partial state observability due to sensor limitations; (b) complex stochastic system with high-dimensional state-spaces, which are jointly continuous and discrete; (c) uncertainties due to ambient weather conditions, occupant's behavior, and comfort feelings. Especially, the partial observability and uncertainty due to the occupant interaction significantly complicate the control problem. Through simulation studies, the policy learned by DDPG demonstrates reasonable performance and computational tractability.