GenCos' Behaviors Modeling Based on Q Learning Improved by Dichotomy

TL;DR

This paper introduces an improved Q learning algorithm using dichotomy to model GenCos' behaviors more efficiently in electricity markets, reducing convergence time through state and action space dichotomization.

Contribution

The paper proposes a novel Q learning method enhanced by dichotomy, significantly improving convergence efficiency in modeling GenCos' behaviors.

Findings

Effective in reducing convergence time

Demonstrated success in repeated Cournot game simulations

Improves practical applicability of Q learning in market modeling

Abstract

Q learning is widely used to simulate the behaviors of generation companies (GenCos) in an electricity market. However, existing Q learning method usually requires numerous iterations to converge, which is time-consuming and inefficient in practice. To enhance the calculation efficiency, a novel Q learning algorithm improved by dichotomy is proposed in this paper. This method modifies the update process of the Q table by dichotomizing the state space and the action space step by step. Simulation results in a repeated Cournot game show the effectiveness of the proposed algorithm.