On Incorporating Forecasts into Linear State Space Model Markov Decision Processes

TL;DR

This paper introduces a novel augmented state space model for Markov decision processes that incorporates forecast information with consistent evolution, enabling more accurate control in energy systems.

Contribution

It presents the first MDP formulation that enforces martingale model for forecast evolution (MMFE) consistency while maintaining computational tractability.

Findings

The model effectively integrates forecast data into MDPs.

It ensures consistent joint evolution of forecasts and states.

The approach is applicable to energy system control.

Abstract

Weather forecast information will very likely find increasing application in the control of future energy systems. In this paper, we introduce an augmented state space model formulation with linear dynamics, within which one can incorporate forecast information that is dynamically revealed alongside the evolution of the underlying state variable. We use the martingale model for forecast evolution (MMFE) to enforce the necessary consistency properties that must govern the joint evolution of forecasts with the underlying state. The formulation also generates jointly Markovian dynamics that give rise to Markov decision processes (MDPs) that remain computationally tractable. This paper is the first to enforce MMFE consistency requirements within an MDP formulation that preserves tractability.