Arbitrage of Energy Storage in Electricity Markets with Deep Reinforcement Learning

TL;DR

This paper proposes a deep reinforcement learning approach to optimize energy storage arbitrage in real-time electricity markets, effectively handling price uncertainty through a stochastic control policy learned via neural networks.

Contribution

It introduces a novel deep reinforcement learning algorithm for real-time energy storage control, formulated as a Markov decision process, and demonstrates its effectiveness with real market data.

Findings

Effective control policy learned for energy storage arbitrage

Successful application to PJM electricity market data

Improved decision-making under price uncertainty

Abstract

In this letter, we address the problem of controlling energy storage systems (ESSs) for arbitrage in real-time electricity markets under price uncertainty. We first formulate this problem as a Markov decision process, and then develop a deep reinforcement learning based algorithm to learn a stochastic control policy that maps a set of available information processed by a recurrent neural network to ESSs' charging/discharging actions. Finally, we verify the effectiveness of our algorithm using real-time electricity prices from PJM.