Energy Storage Arbitrage in Real-Time Markets via Reinforcement Learning

TL;DR

This paper presents a reinforcement learning approach to optimize energy storage arbitrage in real-time markets, effectively handling price uncertainty and improving profitability over existing methods.

Contribution

It introduces a novel reinforcement learning-based policy for energy storage arbitrage that incorporates historical information into the reward function, outperforming traditional strategies.

Findings

Significant performance improvement over existing algorithms.

Effective handling of price uncertainty in arbitrage decisions.

Enhanced profit through history-aware reward design.

Abstract

In this paper, we derive a temporal arbitrage policy for storage via reinforcement learning. Real-time price arbitrage is an important source of revenue for storage units, but designing good strategies have proven to be difficult because of the highly uncertain nature of the prices. Instead of current model predictive or dynamic programming approaches, we use reinforcement learning to design an optimal arbitrage policy. This policy is learned through repeated charge and discharge actions performed by the storage unit through updating a value matrix. We design a reward function that does not only reflect the instant profit of charge/discharge decisions but also incorporate the history information. Simulation results demonstrate that our designed reward function leads to significant performance improvement compared with existing algorithms.