Decentralized Coordination of Distributed Energy Resources through Local Energy Markets and Deep Reinforcement Learning

TL;DR

This paper demonstrates that deep reinforcement learning agents can effectively automate participation in local energy markets, reducing net load variability and energy costs in decentralized power systems.

Contribution

It introduces a DRL-based approach for community-level energy management, addressing variability and scalability issues overlooked by prior socioeconomic-focused studies.

Findings

DRL agents achieve comparable load reduction to dynamic programming benchmarks.

Significant decrease in net load variability metrics like ramping rate and peak demand.

DRL demonstrates scalability and near-optimal performance in decentralized energy markets.

Abstract

As distributed energy resources (DERs) grow, the electricity grid faces increased net load variability at the grid edge, impacting operability and reliability. Transactive energy, facilitated through local energy markets, offers a decentralized, indirect demand response solution, with model-free control techniques, such as deep reinforcement learning (DRL), enabling automated, decentralized participation. However, existing studies largely overlook community-level net load variability, focusing instead on socioeconomic metrics. This study addresses this gap by using DRL agents to automate end-user participation in a local energy market (ALEX), where agents act independently to minimize individual energy bills. Results reveal a strong link between bill reduction and decreased net load variability, assessed across metrics such as ramping rate, load factor, and peak demand over various time horizons. Using a no-control baseline, DRL agents are benchmarked against a near-optimal dynamic programming approach. The dynamic programming benchmark achieves reductions of 22.05 percent, 83.92 percent, and 24.09 percent in daily import, export, and peak demand, respectively, while the DRL agents show comparable or superior results with reductions of 21.93 percent, 84.46 percent, and 27.02 percent. This study demonstrates the effectiveness of DRL in decentralized grid management, highlighting its scalability and near-optimal performance in reducing net load variability within community-driven energy markets.