Multi-Agent Reinforcement Learning for Fast-Timescale Demand Response of Residential Loads

TL;DR

This paper introduces a decentralized multi-agent reinforcement learning approach with communication strategies to improve fast-timescale demand response in residential loads, enhancing grid stability amidst renewable energy variability.

Contribution

It proposes a novel multi-agent reinforcement learning framework with localized communication for demand response, scalable to large residential loads and effective for frequency regulation.

Findings

Policies perform well and robustly for frequency regulation.

Scales seamlessly to arbitrary numbers of houses.

Maintains constant processing times regardless of the number of agents.

Abstract

To integrate high amounts of renewable energy resources, electrical power grids must be able to cope with high amplitude, fast timescale variations in power generation. Frequency regulation through demand response has the potential to coordinate temporally flexible loads, such as air conditioners, to counteract these variations. Existing approaches for discrete control with dynamic constraints struggle to provide satisfactory performance for fast timescale action selection with hundreds of agents. We propose a decentralized agent trained with multi-agent proximal policy optimization with localized communication. We explore two communication frameworks: hand-engineered, or learned through targeted multi-agent communication. The resulting policies perform well and robustly for frequency regulation, and scale seamlessly to arbitrary numbers of houses for constant processing times.