Skolem and Positivity Completeness of Ergodic Markov Chains

TL;DR

This paper establishes a connection between Markov Chain reachability problems and Linear Recurrence Sequences, providing improved reductions that facilitate the analysis of Skolem and Positivity problems in ergodic Markov Chains.

Contribution

It introduces an elementary reduction from LRS problems to ergodic Markov Chains, improving previous mappings by reducing chain complexity and spectral assumptions.

Findings

Mapped LRS to ergodic Markov Chains with spectral structure

Reduced LRS of order k to Markov Chains of order k+1

Improved reduction over previous methods that used reducible chains

Abstract

We consider the following Markov Reachability decision problems that view Markov Chains as Linear Dynamical Systems: given a finite, rational Markov Chain, source and target states, and a rational threshold, does the probability of reaching the target from the source at the $n^{th}$ step: (i) equal the threshold for some $n$? (ii) cross the threshold for some $n$? (iii) cross the threshold for infinitely many $n$? These problems are respectively known to be equivalent to the Skolem, Positivity, and Ultimate Positivity problems for Linear Recurrence Sequences (LRS), number-theoretic problems whose decidability has been open for decades. We present an elementary reduction from LRS Problems to Markov Reachability Problems that improves the state of the art as follows. (a) We map LRS to ergodic (irreducible and aperiodic) Markov Chains that are ubiquitous, not least by virtue of their spectral structure, and (b) our reduction maps LRS of order $k$ to Markov Chains of order $k+1$: a substantial improvement over the previous reduction that mapped LRS of order $k$ to reducible and periodic Markov chains of order $4k+5$. This contribution is significant in view of the fact that the number-theoretic hardness of verifying Linear Dynamical Systems can often be mitigated by spectral assumptions and restrictions on order.