Approximate dynamic programming for profit estimation of connected hydro reservoirs

TL;DR

This paper introduces an approximate dynamic programming method using linear approximation and post-decision states to efficiently estimate profits in connected hydro reservoirs under uncertainty, improving accuracy over traditional methods.

Contribution

It presents a novel approximate dynamic programming approach with linear value function approximation and post-decision states for connected hydro reservoirs, addressing high dimensionality issues.

Findings

Profit estimation improved by 20% with inflow data.

Method provides an upper bound on future value functions.

Demonstrates computational tractability and convergence on realistic systems.

Abstract

In this paper, we study the operational problem of connected hydro power reservoirs which involves sequential decision-making in an uncertain and dynamic environment. The problem is traditionally formulated as a stochastic dynamic program accounting for the uncertainty of electricity prices and reservoir inflows. This formulation suffers from the curse of dimensionality, as the state space explodes with the number of reservoirs and the history of prices and inflows. To avoid computing the expectation of future value functions, the proposed model takes advantage of the so-called post-decision state. To further tackle the dimensionality issue, we propose an approximate dynamic programming approach that estimates the future value of water using a linear approximation architecture. When the time series of prices and inflows follow autoregressive processes, our approximation provides an upper bound on the future value function. We use an offline training algorithm based on the historical data of prices and inflows and run both in-sample and out-of-sample simulations. Two realistic test systems of cascade and network connected reservoirs serve to demonstrate the computational tractability of our approach. In particular, we provide numerical evidence of convergence and quality of solutions. For our test systems, our results show that profit estimation is improved by 20% when including inflows in the linear approximation.