HMPC-assisted Adversarial Inverse Reinforcement Learning for Smart Home Energy Management

TL;DR

This paper introduces a novel energy management approach for smart homes that combines hierarchical model predictive control with adversarial inverse reinforcement learning, eliminating the need for explicit thermal models and manual reward design.

Contribution

It presents a new HMPC-assisted AIRL framework that learns energy management policies without explicit thermal dynamics or predefined rewards.

Findings

Effective in real-world simulations

Data-efficient learning process

Outperforms traditional methods

Abstract

This letter proposes an Adversarial Inverse Reinforcement Learning (AIRL)-based energy management method for a smart home, which incorporates an implicit thermal dynamics model. In the proposed method, historical optimal decisions are first generated using a neural network-assisted Hierarchical Model Predictive Control (HMPC) framework. These decisions are then used as expert demonstrations in the AIRL module, which aims to train a discriminator to distinguish expert demonstrations from transitions generated by a reinforcement learning agent policy, while simultaneously updating the agent policy that can produce transitions to confuse the discriminator. The proposed HMPC-AIRL method eliminates the need for explicit thermal dynamics models, prior or predictive knowledge of uncertain parameters, or manually designed reward functions. Simulation results based on real-world traces demonstrate the effectiveness and data efficiency of the proposed method.