A Reinforcement Learning-based Transmission Expansion Framework Considering Strategic Bidding in Electricity Markets

TL;DR

This paper presents a reinforcement learning framework that co-optimizes transmission expansion and generator bidding strategies in electricity markets, capturing their mutual influence through iterative market simulation.

Contribution

It introduces a novel multiagent RL approach with a design policy layer for joint optimization of transmission expansion and bidding strategies.

Findings

Effective co-optimization of transmission and bidding decisions.

Captures strategic bidding influence on expansion planning.

Validated on IEEE 30-bus system case studies.

Abstract

Transmission expansion planning in electricity markets is tightly coupled with the strategic bidding behaviors of generation companies. This paper proposes a Reinforcement Learning (RL)-based co-optimization framework that simultaneously learns transmission investment decisions and generator bidding strategies within a unified training process. Based on a multiagent RL framework for market simulation, the proposed method newly introduces a design policy layer that jointly optimizes continuous/discrete transmission expansion decisions together with strategic bidding policies. Through iterative interaction between market clearing and investment design, the framework effectively captures their mutual influence and achieves consistent co-optimization of expansion and bidding decisions. Case studies on the IEEE 30-bus system are provided for proof-of-concept validation of the proposed co-optimization framework.