Deep Reinforcement Learning for Heat Pump Control

TL;DR

This paper demonstrates that deep reinforcement learning can be effectively used for heat pump control in buildings, achieving performance comparable to model predictive control without relying on building models.

Contribution

It introduces a model-free DRL approach for heat pump control and provides a detailed comparison with MPC, including analysis of learned control strategies.

Findings

DRL achieves MPC-like performance in heat pump control

DRL operates effectively in a model-free environment

In-depth analysis of control strategies enhances understanding

Abstract

Heating in private households is a major contributor to the emissions generated today. Heat pumps are a promising alternative for heat generation and are a key technology in achieving our goals of the German energy transformation and to become less dependent on fossil fuels. Today, the majority of heat pumps in the field are controlled by a simple heating curve, which is a naive mapping of the current outdoor temperature to a control action. A more advanced control approach is model predictive control (MPC) which was applied in multiple research works to heat pump control. However, MPC is heavily dependent on the building model, which has several disadvantages. Motivated by this and by recent breakthroughs in the field, this work applies deep reinforcement learning (DRL) to heat pump control in a simulated environment. Through a comparison to MPC, it could be shown that it is possible to apply DRL in a model-free manner to achieve MPC-like performance. This work extends other works which have already applied DRL to building heating operation by performing an in-depth analysis of the learned control strategies and by giving a detailed comparison of the two state-of-the-art control methods.