Andes_gym: A Versatile Environment for Deep Reinforcement Learning in Power Systems

TL;DR

Andes_gym is a flexible, high-performance reinforcement learning environment designed for power system studies, integrating ANDES simulation capabilities with OpenAI Gym to facilitate RL algorithm development and testing.

Contribution

It introduces a versatile software tool that combines power system simulation with RL environments, supporting all models in ANDES and RL algorithms in OpenAI Gym.

Findings

Enables rapid prototyping of RL-based load-frequency control.

Supports all dynamic models in ANDES.

Integrates seamlessly with OpenAI Gym for RL experimentation.

Abstract

This paper presents Andes_gym, a versatile and high-performance reinforcement learning environment for power system studies. The environment leverages the modeling and simulation capability of ANDES and the reinforcement learning (RL) environment OpenAI Gym to enable the prototyping and demonstration of RL algorithms for power systems. The architecture of the proposed software tool is elaborated to provide the observation and action interfaces for RL algorithms. An example is shown to rapidly prototype a load-frequency control algorithm based on RL trained by available algorithms. The proposed environment is highly generalized by supporting all the power system dynamic models available in ANDES and numerous RL algorithms available for OpenAI Gym.