Deep Reinforcement Learning for Optimizing Energy Consumption in Smart Grid Systems

TL;DR

This paper introduces a physics-informed neural network surrogate to improve reinforcement learning efficiency for energy management in smart grids, reducing training time and eliminating the need for costly simulators.

Contribution

It presents a novel PINN-based surrogate model that accelerates RL training and maintains accuracy without requiring samples from the actual smart grid simulator.

Findings

PINN surrogate reduces RL training time by 50%.

The approach achieves comparable performance scores to the original simulator.

PINN surrogate can learn effective policies without access to true simulator samples.

Abstract

The energy management problem in the context of smart grids is inherently complex due to the interdependencies among diverse system components. Although Reinforcement Learning (RL) has been proposed for solving Optimal Power Flow (OPF) problems, the requirement for iterative interaction with an environment often necessitates computationally expensive simulators, leading to significant sample inefficiency. In this study, these challenges are addressed through the use of Physics-Informed Neural Networks (PINNs), which can replace conventional and costly smart grid simulators. The RL policy learning process is enhanced so that convergence can be achieved in a fraction of the time required by the original environment. The PINN-based surrogate is compared with other benchmark data-driven surrogate models. By incorporating knowledge of the underlying physical laws, the results show that the PINN surrogate is the only approach considered in this context that can obtain a strong RL policy even without access to samples from the true simulator. The results demonstrate that using PINN surrogates can accelerate training by 50% compared to RL training without a surrogate. This approach enables the rapid generation of performance scores similar to those produced by the original simulator.