Multi-agent Deep Reinforcement Learning for Distributed Load Restoration

TL;DR

This paper introduces a multi-agent deep reinforcement learning approach with invalid action masking for efficient, constraint-compliant load restoration in distribution systems, demonstrating superior performance over traditional methods.

Contribution

It presents a novel multi-agent DRL framework with invalid action masking, centralized training, and decentralized execution for distribution load restoration.

Findings

Achieves better learning stability than conventional methods

Zero constraint violations in restoration actions

Effective in large-scale distribution test feeders

Abstract

This paper addresses the load restoration problem after power outage events. Our primary proposed methodology is using multi-agent deep reinforcement learning to optimize the load restoration process in distribution systems, modeled as networked microgrids, via determining the optimal operational sequence of circuit breakers (switches). An innovative invalid action masking technique is incorporated into the multi-agent method to handle both the physical constraints in the restoration process and the curse of dimensionality as the action space of operational decisions grows exponentially with the number of circuit breakers. The features of our proposed method include centralized training for multi-agents to overcome non-stationary environment problems, decentralized execution to ease the deployment, and zero constraint violations to prevent harmful actions. Our simulations are performed in OpenDSS and Python environments to demonstrate the effectiveness of the proposed approach using the IEEE 13, 123, and 8500-node distribution test feeders. The results show that the proposed algorithm can achieve a significantly better learning curve and stability than the conventional methods.