Safe multi-agent deep reinforcement learning for joint bidding and maintenance scheduling of generation units

TL;DR

This paper introduces a safe deep reinforcement learning algorithm for joint bidding and maintenance scheduling in electricity markets, effectively handling safety constraints and incomplete information to improve profitability.

Contribution

It develops a novel safe deep deterministic policy gradient algorithm combining reinforcement learning with a safety filter for complex multi-agent decision-making.

Findings

Achieves higher profit than existing methods.

Successfully maintains safety constraints during operation.

Handles incomplete information among agents.

Abstract

This paper proposes a safe reinforcement learning algorithm for generation bidding decisions and unit maintenance scheduling in a competitive electricity market environment. In this problem, each unit aims to find a bidding strategy that maximizes its revenue while concurrently retaining its reliability by scheduling preventive maintenance. The maintenance scheduling provides some safety constraints which should be satisfied at all times. Satisfying the critical safety and reliability constraints while the generation units have an incomplete information of each others' bidding strategy is a challenging problem. Bi-level optimization and reinforcement learning are state of the art approaches for solving this type of problems. However, neither bi-level optimization nor reinforcement learning can handle the challenges of incomplete information and critical safety constraints. To tackle these challenges, we propose the safe deep deterministic policy gradient reinforcement learning algorithm which is based on a combination of reinforcement learning and a predicted safety filter. The case study demonstrates that the proposed approach can achieve a higher profit compared to other state of the art methods while concurrently satisfying the system safety constraints.