Scalable Grid-Aware Dynamic Matching using Deep Reinforcement Learning

TL;DR

This paper introduces a hierarchical framework using deep reinforcement learning for scalable, grid-aware matching of renewable energy sources and flexible customers, optimizing power flow while respecting grid constraints.

Contribution

It presents a novel two-level hierarchical matching approach combining deep reinforcement learning and optimal power flow to improve efficiency and scalability in grid-aware resource matching.

Findings

Framework effectively matches resources while respecting grid constraints.

Deep RL agents successfully identify optimal matching solutions.

Case studies demonstrate improved welfare and service satisfaction.

Abstract

This paper proposes a two-level hierarchical matching framework for Integrated Hybrid Resources (IHRs) with grid constraints. An IHR is a collection of Renewable Energy Sources (RES) and flexible customers within a certain power system zone, endowed with an agent to match. The key idea is to pick the IHR zones so that the power loss effects within the IHRs can be neglected. This simplifies the overall matching problem into independent IHR-level matching problems and an upper-level optimal power flow problem to meet the IHR-level upstream flow requirements while respecting the grid constraints. Within each IHR, the agent employs a scalable Deep Reinforcement Learning algorithm to identify matching solutions such that the customer's service constraints are met. The central agent then solves an optimal power flow problem with the IHRs as the nodes, with their active power flow and reactive power {capacities}, and grid constraints to scalably determine the final flows such that matched power can be delivered to the extent the grid constraints are satisfied. The proposed framework is implemented on a test power distribution system, and multiple case studies are presented to substantiate the welfare efficiency of the proposed solution and the satisfaction of the grid and customers' servicing constraints.