Deep Learning-based Predictive Control of Battery Management for Frequency Regulation

TL;DR

This paper introduces a deep learning-based battery management scheme for frequency regulation that combines model predictive control, supervised learning, and reinforcement learning to improve economic benefits and computational efficiency.

Contribution

It presents a novel two-stage training process integrating SL and RL to optimize battery management policies using high-fidelity models.

Findings

Achieves higher economic benefits compared to conventional MPC.

Maintains lower online computational cost.

Effectively learns from high-fidelity battery simulations.

Abstract

This paper proposes a deep learning-based optimal battery management scheme for frequency regulation (FR) by integrating model predictive control (MPC), supervised learning (SL), reinforcement learning (RL), and high-fidelity battery models. By taking advantage of deep neural networks (DNNs), the derived DNN-approximated policy is computationally efficient in online implementation. The design procedure of the proposed scheme consists of two sequential processes: (1) the SL process, in which we first run a simulation with an MPC embedding a low-fidelity battery model to generate a training data set, and then, based on the generated data set, we optimize a DNN-approximated policy using SL algorithms; and (2) the RL process, in which we utilize RL algorithms to improve the performance of the DNN-approximated policy by balancing short-term economic incentives and long-term battery degradation. The SL process speeds up the subsequent RL process by providing a good initialization. By utilizing RL algorithms, one prominent property of the proposed scheme is that it can learn from the data generated by simulating the FR policy on the high-fidelity battery simulator to adjust the DNN-approximated policy, which is originally based on low-fidelity battery model. A case study using real-world data of FR signals and prices is performed. Simulation results show that, compared to conventional MPC schemes, the proposed deep learning-based scheme can effectively achieve higher economic benefits of FR participation while maintaining lower online computational cost.