Electric Arc Furnaces Scheduling under Electricity Price Volatility with Reinforcement Learning

TL;DR

This paper introduces a reinforcement learning framework for optimizing electric arc furnace operations amid volatile electricity prices, demonstrating near-optimal profit performance in real-world scenarios.

Contribution

It develops a Q-learning-based control method for EAF scheduling that handles price volatility and capacity constraints, outperforming rule-based controllers.

Findings

Achieves around 90% of the profit of an ideal MILP benchmark.

Effectively manages real-time electricity price fluctuations.

Outperforms rule-based control in various scenarios.

Abstract

This paper proposes a reinforcement learning-based framework for optimizing the operation of electric arc furnaces (EAFs) under volatile electricity prices. We formulate the deterministic version of the EAF scheduling problem into a mixed-integer linear programming (MILP) formulation, and then develop a Q-learning algorithm to perform real-time control of multiple EAF units under real-time price volatility and shared feeding capacity constraints. We design a custom reward function for the Q-learning algorithm to smooth the start-up penalties of the EAFs. Using real data from EAF designs and electricity prices in New York State, we benchmark our algorithm against a baseline rule-based controller and a MILP benchmark, assuming perfect price forecasts. The results show that our reinforcement learning algorithm achieves around 90% of the profit compared to the perfect MILP benchmark in various single-unit and multi-unit cases under a non-anticipatory control setting.