Reinforcement Learning for Bidding Strategy Optimization in Day-Ahead Energy Market

TL;DR

This paper develops a reinforcement learning-based bidding strategy for energy sellers in day-ahead markets, aiming to optimize profits by controlling bidding curves using historical market and cost data.

Contribution

It introduces a novel RL approach with Deep Deterministic Policy Gradient for continuous bidding strategies in energy markets, focusing on optimizing individual payoffs.

Findings

RL-based bidding improves market participation efficiency

The approach adapts to market dynamics using historical data

Enhanced profit maximization for energy sellers

Abstract

In a day-ahead market, energy buyers and sellers submit their bids for a particular future time, including the amount of energy they wish to buy or sell and the price they are prepared to pay or receive. However, the dynamic for forming the Market Clearing Price (MCP) dictated by the bidding mechanism is frequently overlooked in the literature on energy market modelling. Forecasting models usually focus on predicting the MCP rather than trying to build the optimal supply and demand curves for a given price scenario. Following this approach, the article focuses on developing a bidding strategy for a seller in a continuous action space through a single agent Reinforcement Learning algorithm, specifically the Deep Deterministic Policy Gradient. The algorithm controls the offering curve (action) based on past data (state) to optimize future payoffs (rewards). The participant can access historical data on production costs, capacity, and prices for various sources, including renewable and fossil fuels. The participant gains the ability to operate in the market with greater efficiency over time to maximize individual payout.