Probabilistic Reachability Analysis of Multi-scale Voltage Dynamics Using Reinforcement Learning

TL;DR

This paper introduces a deep reinforcement learning framework for probabilistic reachability analysis of multi-scale voltage dynamics, effectively identifying and quantifying instability risks in power systems with reduced computational effort.

Contribution

It proposes a novel RL-based method that models multiple instability mechanisms as absorbing states with a multi-critic architecture for risk assessment.

Findings

Successfully applied to a four-bus system with load tap changers.

Effectively identifies mechanisms leading to voltage collapse.

Reduces computational cost compared to Monte Carlo simulations.

Abstract

Voltage stability in modern power systems involves coupled dynamics across multiple time scales. Conventional methods based on time-scale separation or static stability margins may overlook instabilities caused by the coupling of slow and fast transients. Uncertainty in operating conditions further complicates stability assessment, and high computational cost of Monte Carlo simulations limit its applicability to multi-scale dynamics. This paper presents a deep reinforcement learning-based framework for probabilistic reachability analysis of multi-scale voltage dynamics. By formulating each instability mechanism as a distinct absorbing state and introducing a multi-critic architecture for mechanism-specific learning, the proposed method enables consistent learning of risk probabilities associated with multiple instability types within a unified framework. The approach is demonstrated on a four-bus system with load tap changers and over-excitation limiters, illustrating effectiveness of the proposed learning-based reachability analysis in identifying and quantifying the mechanisms leading to voltage collapse.