Controlling Commercial Cooling Systems Using Reinforcement Learning

TL;DR

This paper discusses the application of reinforcement learning to control commercial cooling systems, highlighting real-world experiments that achieved significant energy savings despite various challenges.

Contribution

It presents practical methods for applying RL to real-world cooling systems and reports successful energy savings from live experiments.

Findings

Energy savings of approximately 9% and 13% at two sites

Addressed challenges in evaluation, offline learning, and constraints

Demonstrated feasibility of RL in commercial building management

Abstract

This paper is a technical overview of DeepMind and Google's recent work on reinforcement learning for controlling commercial cooling systems. Building on expertise that began with cooling Google's data centers more efficiently, we recently conducted live experiments on two real-world facilities in partnership with Trane Technologies, a building management system provider. These live experiments had a variety of challenges in areas such as evaluation, learning from offline data, and constraint satisfaction. Our paper describes these challenges in the hope that awareness of them will benefit future applied RL work. We also describe the way we adapted our RL system to deal with these challenges, resulting in energy savings of approximately 9% and 13% respectively at the two live experiment sites.