Multi-weighted Markov Decision Processes with Reachability Objectives

TL;DR

This paper develops methods for synthesizing schedulers in double-weighted Markov decision processes that satisfy both percentile and expected value constraints related to reachability, inspired by electric vehicle charging.

Contribution

It introduces a novel approach to handle combined percentile and expectation constraints in double-weighted MDPs, including partial cartography and optimization techniques.

Findings

Partial cartography of the problem space obtained.

Method's completeness and feasibility discussed.

Application to electric vehicle charging modeled.

Abstract

In this paper, we are interested in the synthesis of schedulers in double-weighted Markov decision processes, which satisfy both a percentile constraint over a weighted reachability condition, and a quantitative constraint on the expected value of a random variable defined using a weighted reachability condition. This problem is inspired by the modelization of an electric-vehicle charging problem. We study the cartography of the problem, when one parameter varies, and show how a partial cartography can be obtained via two sequences of opimization problems. We discuss completeness and feasability of the method.