Deep Reinforcement Learning for Wind and Energy Storage Coordination in Wholesale Energy and Ancillary Service Markets

TL;DR

This paper presents a deep reinforcement learning approach for co-located wind and battery systems to optimize market participation, reduce wind curtailment, and increase revenue, outperforming traditional optimization methods.

Contribution

It introduces a novel RL-based joint-market bidding strategy that decouples decision processes for wind and BESS, enhancing revenue and curtailment reduction.

Findings

Revenues increased by approximately 25% compared to benchmarks.

Wind curtailment reduced by 2.3 times using the proposed strategy.

Joint-market bidding outperforms separate market participation.

Abstract

Wind energy has been increasingly adopted to mitigate climate change. However, the variability of wind energy causes wind curtailment, resulting in considerable economic losses for wind farm owners. Wind curtailment can be reduced using battery energy storage systems (BESS) as onsite backup sources. Yet, this auxiliary role may significantly weaken the economic potential of BESS in energy trading. Ideal BESS scheduling should balance onsite wind curtailment reduction and market bidding, but practical implementation is challenging due to coordination complexity and the stochastic nature of energy prices and wind generation. We investigate the joint-market bidding strategy of a co-located wind-battery system in the spot and Regulation Frequency Control Ancillary Service markets. We propose a novel deep reinforcement learning-based approach that decouples the system's market participation into two related Markov decision processes for each facility, enabling the BESS to absorb onsite wind curtailment while performing joint-market bidding to maximize overall operational revenues. Using realistic wind farm data, we validated the coordinated bidding strategy, with outcomes surpassing the optimization-based benchmark in terms of higher revenue by approximately 25\% and more wind curtailment reduction by 2.3 times. Our results show that joint-market bidding can significantly improve the financial performance of wind-battery systems compared to participating in each market separately. Simulations also show that using curtailed wind generation as a power source for charging the BESS can lead to additional financial gains. The successful implementation of our algorithm would encourage co-location of generation and storage assets to unlock wider system benefits.