Optimizing Performance of Continuous-Time Stochastic Systems using Timeout Synthesis

TL;DR

This paper presents a method to optimize parameters in continuous-time stochastic systems, specifically fixed-delay Markov chains, by translating the problem into a Markov decision process to efficiently approximate optimal parameter values.

Contribution

It introduces a novel approach for parameter synthesis in parametric fixed-delay Markov chains using MDP translation for effective approximation.

Findings

Effective approximation of optimal parameters achieved

Method applicable under mild assumptions

Potential for improved system performance

Abstract

We consider parametric version of fixed-delay continuous-time Markov chains (or equivalently deterministic and stochastic Petri nets, DSPN) where fixed-delay transitions are specified by parameters, rather than concrete values. Our goal is to synthesize values of these parameters that, for a given cost function, minimise expected total cost incurred before reaching a given set of target states. We show that under mild assumptions, optimal values of parameters can be effectively approximated using translation to a Markov decision process (MDP) whose actions correspond to discretized values of these parameters.