Controlling a random population

TL;DR

This paper proves the decidability of the stochastic control problem for parameterised systems of agents modeled as Markov decision processes, using advanced mathematical tools.

Contribution

It introduces a decidability result for the stochastic control problem and develops the sequential flow problem as a new complexity-related concept.

Findings

Decidability of the stochastic control problem for parameterised systems.

Development of the sequential flow problem and analysis of its complexity.

Application of well quasi orders, max-flow min-cut theorem, and regular cost functions.

Abstract

Bertrand et al. introduced a model of parameterised systems, where each agent is represented by a finite state system, and studied the following control problem: for any number of agents, does there exist a controller able to bring all agents to a target state? They showed that the problem is decidable and EXPTIME-complete in the adversarial setting, and posed as an open problem the stochastic setting, where the agent is represented by a Markov decision process. In this paper, we show that the stochastic control problem is decidable. Our solution makes significant uses of well quasi orders, of the max-flow min-cut theorem, and of the theory of regular cost functions. We introduce an intermediate problem of independence interest called the sequential flow problem and study its complexity.