Reachability analysis in stochastic directed graphs by reinforcement learning

TL;DR

This paper introduces a reinforcement learning-based approach to analyze reachability probabilities in stochastic directed graphs, with applications to epidemic diffusion modeling on dynamic contact networks.

Contribution

It models transition dynamics via difference inclusion as a Markov decision process and designs reward functions to bound reachability probabilities in stochastic digraphs.

Findings

Effective bounds on reachability probabilities demonstrated

Application to epidemic spread on mobile agent networks

Method shows promise for dynamic network analysis

Abstract

We characterize the reachability probabilities in stochastic directed graphs by means of reinforcement learning methods. In particular, we show that the dynamics of the transition probabilities in a stochastic digraph can be modeled via a difference inclusion, which, in turn, can be interpreted as a Markov decision process. Using the latter framework, we offer a methodology to design reward functions to provide upper and lower bounds on the reachability probabilities of a set of nodes for stochastic digraphs. The effectiveness of the proposed technique is demonstrated by application to the diffusion of epidemic diseases over time-varying contact networks generated by the proximity patterns of mobile agents.