Scalable Multi-Agent Reinforcement Learning for Residential Load Scheduling under Data Governance

TL;DR

This paper introduces a scalable multi-agent reinforcement learning framework for residential load scheduling that preserves privacy and reduces communication costs, enabling practical deployment in privacy-sensitive, communication-constrained environments.

Contribution

It proposes a novel actor-critic MARL approach with a global critic based on local critics, addressing privacy and communication challenges in large-scale residential load scheduling.

Findings

Achieves comparable performance to state-of-the-art methods without privacy constraints.

Significantly reduces communication overhead during coordination.

Maintains household privacy completely in distributed training.

Abstract

As a data-driven approach, multi-agent reinforcement learning (MARL) has made remarkable advances in solving cooperative residential load scheduling problems. However, centralized training, the most common paradigm for MARL, limits large-scale deployment in communication-constrained cloud-edge environments. As a remedy, distributed training shows unparalleled advantages in real-world applications but still faces challenge with system scalability, e.g., the high cost of communication overhead during coordinating individual agents, and needs to comply with data governance in terms of privacy. In this work, we propose a novel MARL solution to address these two practical issues. Our proposed approach is based on actor-critic methods, where the global critic is a learned function of individual critics computed solely based on local observations of households. This scheme preserves household privacy completely and significantly reduces communication cost. Simulation experiments demonstrate that the proposed framework achieves comparable performance to the state-of-the-art actor-critic framework without data governance and communication constraints.