A Meta-Learning Approach for Multi-Objective Reinforcement Learning in Sustainable Home Environments

TL;DR

This paper introduces a meta-learning enhanced multi-objective reinforcement learning approach for residential appliance scheduling, enabling rapid adaptation to changing energy environments and improving cost, comfort, and utility with less data.

Contribution

It extends MORL with meta-learning and an AE-based context detection method, demonstrating significant performance improvements in real-world residential energy management.

Findings

Model saves 3.28% on electricity bills

Achieves 2.74% higher user comfort

Reduces solution sparsity by 62.44%

Abstract

Effective residential appliance scheduling is crucial for sustainable living. While multi-objective reinforcement learning (MORL) has proven effective in balancing user preferences in appliance scheduling, traditional MORL struggles with limited data in non-stationary residential settings characterized by renewable generation variations. Significant context shifts that can invalidate previously learned policies. To address these challenges, we extend state-of-the-art MORL algorithms with the meta-learning paradigm, enabling rapid, few-shot adaptation to shifting contexts. Additionally, we employ an auto-encoder (AE)-based unsupervised method to detect environment context changes. We have also developed a residential energy environment to evaluate our method using real-world data from London residential settings. This study not only assesses the application of MORL in residential appliance scheduling but also underscores the effectiveness of meta-learning in energy management. Our top-performing method significantly surpasses the best baseline, while the trained model saves 3.28% on electricity bills, a 2.74% increase in user comfort, and a 5.9% improvement in expected utility. Additionally, it reduces the sparsity of solutions by 62.44%. Remarkably, these gains were accomplished using 96.71% less training data and 61.1% fewer training steps.