# Macro-action Multi-time scale Dynamic Programming for Energy Management in Buildings with Phase Change Materials

## TL;DR

This paper introduces a novel macro-action multi-time scale dynamic programming approach for energy management in buildings with phase change materials, significantly reducing computational complexity while optimizing HVAC scheduling.

## Contribution

It develops a new methodology combining macro actions and multi-time scale Markov decision processes to efficiently solve nonlinear energy management problems in buildings.

## Key findings

- Achieves up to 12,900 times speed-up over traditional DP methods.
- Effectively manages nonlinear and non-convex PCM characteristics.
- Demonstrates practical applicability in residential building energy management.

## Abstract

This paper focuses on energy management in buildings with phase change material (PCM), which is primarily used to improve thermal performance, but can also serve as an energy storage system. In this setting, optimal scheduling of an HVAC system is challenging because of the nonlinear and non-convex characteristics of the PCM, which makes solving the corresponding optimization problem using conventional optimization techniques impractical. Instead, we use dynamic programming (DP) to deal with the nonlinear nature of the PCM. To overcome DP's curse of dimensionality, this paper proposes a novel methodology to reduce the computational burden, while maintaining the quality of the solution. Specifically, the method incorporates approaches from sequential decision making in artificial intelligence, including macro actions and multi-time scale Markov decision processes, coupled with an underlying state-space approximation to reduce the state-space and action-space size. The performance of the method is demonstrated on an energy management problem for a typical residential building located in Sydney, Australia. The results demonstrate that the proposed method performs well with a computational speed-up of up to 12,900 times compared to the direct application of DP.