Data-driven control of room temperature and bidirectional EV charging using deep reinforcement learning: simulations and experiments

TL;DR

This paper introduces a data-driven deep reinforcement learning approach for joint control of room temperature and EV charging, achieving energy savings and comfort improvements in simulations and real building experiments.

Contribution

It presents a novel black-box pipeline using neural networks and DRL for building and EV control without complex physics models, validated through simulations and real-world deployment.

Findings

17% energy savings over heating season

19% better comfort satisfaction than standard controllers

30% energy savings in real building experiment

Abstract

This work presents a fully data-driven, black-box pipeline to obtain an optimal control policy for a multi-loop building control problem based on historical building and weather data, thus without the need for complex physics-based modelling. We demonstrate the method for joint control of room temperature and bidirectional EV charging to maximize the occupant thermal comfort and energy savings while leaving enough energy in the EV battery for the next trip. We modelled the room temperature with a recurrent neural network and EV charging with a piece-wise linear function. Using these models as a simulation environment, we applied a deep reinforcement learning (DRL) algorithm to obtain an optimal control policy. The learnt policy achieves on average 17% energy savings over the heating season and 19% better comfort satisfaction than a standard RB room temperature controller. When a bidirectional EV is additionally connected and a two-tariff electricity pricing is applied, the MIMO DRL policy successfully leverages the battery and decreases the overall cost of electricity compared to two standard RB controllers, one controlling the room temperature and another controlling the bidirectional EV (dis-)charging. Finally, we demonstrate a successful transfer of the learnt DRL policy from simulation onto a real building, the DFAB HOUSE at Empa Duebendorf in Switzerland, achieving up to 30% energy savings while maintaining similar comfort levels compared to a conventional RB room temperature controller over three weeks during the heating season.