Expected Window Mean-Payoff

TL;DR

This paper introduces methods to compute the expected supremum of window mean-payoff values in Markov chains and decision processes, extending prior work on ensuring minimum payoffs in two-player games.

Contribution

It defines and analyzes the expected value of the window mean-payoff function in stochastic models, including fixed and bounded window variants, with direct and prefix-independent versions.

Findings

Provides algorithms for expected window mean-payoff in Markov chains and MDPs.

Extends previous work on deterministic strategies to stochastic settings.

Analyzes both fixed and variable window length scenarios.

Abstract

In the window mean-payoff objective, given an infinite path, instead of considering a long run average, we consider the minimum payoff that can be ensured at every position of the path over a finite window that slides over the entire path. Chatterjee et al. studied the problem to decide if in a two-player game, Player 1 has a strategy to ensure a window mean-payoff of at least 0. In this work, we consider a function that given a path returns the supremum value of the window mean-payoff that can be ensured over the path and we show how to compute its expected value in Markov chains and Markov decision processes. We consider two variants of the function: Fixed window mean-payoff in which a fixed window length $l_{max}$ is provided; and Bounded window mean-payoff in which we compute the maximum possible value of the window mean-payoff over all possible window lengths. Further, for both variants, we consider (i) a direct version of the problem where for each path, the payoff that can be ensured from its very beginning and (ii) a non-direct version that is the prefix independent counterpart of the direct version of the problem.