A Hierarchical MARL-Based Approach for Coordinated Retail P2P Trading and Wholesale Market Participation of DERs

TL;DR

This paper introduces a hierarchical MARL framework enabling prosumers to participate in P2P retail and wholesale markets, improving grid flexibility and market efficiency through intelligent DER coordination.

Contribution

It presents a novel hierarchical MARL-based approach combined with a Stackelberg game for coordinated DER market participation.

Findings

Hierarchical MARL effectively manages prosumer participation in P2P and wholesale markets.

The framework enhances market performance and grid operational flexibility.

Simulation results demonstrate improved coordination among DERs.

Abstract

The ongoing shift towards decentralization of the electric energy sector, driven by the growing electrification across end-use sectors, and widespread adoption of distributed energy resources (DERs), necessitates their active participation in the electricity markets to support grid operations. Furthermore, with bi-directional energy and communication flows becoming standard, intelligent, easy-to-deploy, resource-conservative demand-side participation is expected to play a critical role in securing power grid operational flexibility and market efficiency. This work proposes a market engagement framework that leverages a hierarchical multi-agent deep reinforcement learning (MARL) approach to enable individual prosumers to participate in peer-to-peer retail auctions and further aggregate these intelligent prosumers to facilitate effective DER participation in wholesale markets. Ultimately, a Stackelberg game is proposed to coordinate this hierarchical MARL-based DER market participation framework toward enhanced market performance.