Reinforcement Learning for Battery Energy Storage Dispatch augmented with Model-based Optimizer

TL;DR

This paper introduces a hybrid approach combining physics-based models with imitation-enhanced deep reinforcement learning to improve battery dispatch in power distribution systems, addressing scalability and performance issues.

Contribution

It presents a novel imitation learning method that leverages linearized model solutions to enhance DRL for distribution-level battery dispatch optimization.

Findings

Improved training efficiency of DRL algorithms.

Effective dispatch solutions for IEEE distribution feeders.

Scalability in complex power systems.

Abstract

Reinforcement learning has been found useful in solving optimal power flow (OPF) problems in electric power distribution systems. However, the use of largely model-free reinforcement learning algorithms that completely ignore the physics-based modeling of the power grid compromises the optimizer performance and poses scalability challenges. This paper proposes a novel approach to synergistically combine the physics-based models with learning-based algorithms using imitation learning to solve distribution-level OPF problems. Specifically, we propose imitation learning based improvements in deep reinforcement learning (DRL) methods to solve the OPF problem for a specific case of battery storage dispatch in the power distribution systems. The proposed imitation learning algorithm uses the approximate optimal solutions obtained from a linearized model-based OPF solver to provide a good initial policy for the DRL algorithms while improving the training efficiency. The effectiveness of the proposed approach is demonstrated using IEEE 34-bus and 123-bus distribution feeders with numerous distribution-level battery storage systems.