Comparing Traditional and Reinforcement-Learning Methods for Energy Storage Control

TL;DR

This paper compares traditional and reinforcement learning methods for energy storage control in micro-grids, analyzing performance tradeoffs across different storage and loss scenarios to guide method selection.

Contribution

It provides a detailed comparison of traditional and RL approaches for energy storage management in micro-grids, highlighting their respective advantages and limitations.

Findings

RL methods perform comparably to traditional methods in ideal scenarios.

Performance loss of RL increases with storage losses and transmission inefficiencies.

Guidelines for choosing between traditional and RL methods based on system complexity.

Abstract

We aim to better understand the tradeoffs between traditional and reinforcement learning (RL) approaches for energy storage management. More specifically, we wish to better understand the performance loss incurred when using a generative RL policy instead of using a traditional approach to find optimal control policies for specific instances. Our comparison is based on a simplified micro-grid model, that includes a load component, a photovoltaic source, and a storage device. Based on this model, we examine three use cases of increasing complexity: ideal storage with convex cost functions, lossy storage devices, and lossy storage devices with convex transmission losses. With the aim of promoting the principled use RL based methods in this challenging and important domain, we provide a detailed formulation of each use case and a detailed description of the optimization challenges. We then compare the performance of traditional and RL methods, discuss settings in which it is beneficial to use each method, and suggest avenues for future investigation.