Deep Reinforcement Learning for Long Term Hydropower Production Scheduling

TL;DR

This paper investigates the application of deep reinforcement learning, specifically the soft actor-critic algorithm, to optimize long-term hydropower scheduling by balancing immediate revenue against future potential, demonstrating promising results in a simplified scenario.

Contribution

It introduces a reinforcement learning approach for hydropower scheduling, highlighting its potential to complement traditional optimization methods in data-rich environments.

Findings

Successful training of a soft actor-critic model on historical Nordic power data

Reinforcement learning shows potential as a complementary tool for hydropower scheduling

Model demonstrates feasibility but is not yet ready to replace traditional optimization methods

Abstract

We explore the use of deep reinforcement learning to provide strategies for long term scheduling of hydropower production. We consider a use-case where the aim is to optimise the yearly revenue given week-by-week inflows to the reservoir and electricity prices. The challenge is to decide between immediate water release at the spot price of electricity and storing the water for later power production at an unknown price, given constraints on the system. We successfully train a soft actor-critic algorithm on a simplified scenario with historical data from the Nordic power market. The presented model is not ready to substitute traditional optimisation tools but demonstrates the complementary potential of reinforcement learning in the data-rich field of hydropower scheduling.