Non-deterministic weighted automata evaluated over Markov chains

TL;DR

This paper introduces a probabilistic semantics for non-deterministic weighted automata over Markov chains, providing approximation algorithms for key probabilistic questions and demonstrating effective determinisation under certain metrics.

Contribution

It is the first to study non-deterministic weighted automata with probabilistic semantics and offers approximation methods for their probabilistic analysis.

Findings

Approximation algorithms for expected value and distribution of automata outputs.

Automata can be effectively determinised with respect to the standard deviation metric.

Probabilistic questions are uncomputable in general but approximable in exponential time.

Abstract

We present the first study of non-deterministic weighted automata under probabilistic semantics. In this semantics words are random events, generated by a Markov chain, and functions computed by weighted automata are random variables. We consider the probabilistic questions of computing the expected value and the cumulative distribution for such random variables. The exact answers to the probabilistic questions for non-deterministic automata can be irrational and are uncomputable in general. To overcome this limitation, we propose approximation algorithms for the probabilistic questions, which work in exponential time in the size of the automaton and polynomial time in the size of the Markov chain and the given precision. We apply this result to show that non-deterministic automata can be effectively determinised with respect to the standard deviation metric.

Equations

$$\text{textsc}Sum(\pi )={\sum }_{i=1}^{|C(\pi )|}(C(\pi ))[i]$$

$$\text{textsc}LimAvg(\pi )=\underset{k\to \infty }{\mathrm{limsup}}\text{textsc}Avg(\pi [0,k])$$

$${\mathbb{E}}_{\mathcal{M}}(g)=\sum _{y\in Y}y\cdot {\mathbb{P}}_{\mathcal{M}}(\{w\mid g(w)=y\}).$$

$$q_1^w\dots q_1^w{q}_x{q}_2^w\dots q_2^w{q}_x{q}_3^w\dots q_3^w{q}_x{q}_4^w\dots$$

$$\text{textsc}Avg({\pi }_w[0,s_i])\le \text{textsc}Avg(\pi [0,s_i]).$$

$$\text{textsc}Avg({\pi }_w[0,p])\le \text{textsc}Avg(\pi [0,s_i])+\frac{p-s_i}{p}$$

$$\begin{array}{rl}\text{textsc}Avg({\pi }_w[0,p])=\frac{\text{textsc}Sum({\pi }_w[0,p])}{p}& \le \frac{\text{textsc}Sum({\pi }_w[0,s_i])}{s_i}+\frac{\text{textsc}Sum({\pi }_w[s_i,p])}{p}\\ & =\text{textsc}Avg({\pi }_w[0,s_i])+\frac{\text{textsc}Sum({\pi }_w[s_i,p])}{p}.\end{array}$$

$$\text{textsc}Avg({\pi }_w[0,s_i])+\frac{\text{textsc}Sum({\pi }_w[s_i,p])}{p}\le \text{textsc}Avg(\pi [0,s_i])+\frac{p-s_i}{p},$$

$${\mathbb{E}}_{\mathcal{M}}(\text{textsc}LimAvg(\{{\pi }_w\mid w\in X\}))=\underset{s\to \infty }{\mathrm{limsup}}\frac{1}{s}\cdot \sum _{i=1}^s{\mathbb{E}}_{\mathcal{M}}\left(\{(C({\pi }_w))[i]\mid w\in X\}\right)$$

$$\sum _{j=1}^{\infty }\sum _{k=1}^{\infty }\frac{j+k}{2^{j+k+2}}\cdot \frac{\min (j,k)}{j+k}=\sum _{j=1}^{\infty }\sum _{k=1}^{\infty }\frac{\min (j,k)}{2^{j+k+2}}=\frac{1}{3}$$

$$0<\gamma -\frac{p_i}{q_i}<2\cdot 3^{-(l_{i+1}+1)}$$

$$\gamma -\frac{p_i}{q_i}<2\cdot 3^{-(l_{i+1}+1)}<\frac{2}{3}{3}^{-3(l_i+1)}<\frac{1}{q_i^3}.$$

$$\underset{s\to \infty }{\mathrm{liminf}}\frac{1}{s}(X_1+\dots +X_s)=\underset{s\to \infty }{\mathrm{limsup}}\frac{1}{s}(X_1+\dots +X_s)=\mathbb{E}(X_i)=\mathbb{E}(\mathcal{A})$$

$$|{\mathbb{D}}_{{\mathcal{M}}^T,\mathcal{A}}(\lambda )-{\mathbb{D}}_{{\mathcal{M}}^T,\mathcal{A}}(\lambda ,n)|\le \frac{ϵ}{2}.$$

$$\Bigl{\lvert}\mathbb{D}_{\mathcal{M}^{T},\mathcal{A}}(\lambda,n)-\frac{c_{A}}{2^{K}}\Bigl{\rvert}\leq\frac{\epsilon}{2}.$$

$$\Bigl{\lvert}$$

$$c_A=\sum _{w:|w|\le N\wedge {\mathcal{L}}_{\mathcal{A}}(w)\le \lambda }{c}_A^w.$$

$$\Bigl{\lvert}\mathbb{D}_{\mathcal{M}^{T},\mathcal{A}}(\lambda,n)-\frac{c_{A}}{2^{K}}\Bigr{\rvert}\leq\sum_{w\colon|w|\leq N\ \wedge\mathcal{L}_{{\mathcal{A}}}(w)\leq\lambda}\Bigr{\lvert}$$

$$\bigl{\lvert}\mathbb{E}_{\mathcal{M}^{T}}(\mathcal{A})-\mathbb{E}_{\mathcal{M}^{T}}(\mathcal{A},M)\bigl{\rvert}\leq\frac{\epsilon}{2}.$$

$$\bigl{\lvert}\mathbb{E}_{\mathcal{M}^{T}}(\mathcal{A}^{\prime},M)-\frac{c_{A}}{2^{K}}\bigl{\rvert}\leq\frac{\epsilon}{2}.$$

$$\bigl{\lvert}\mathbb{E}_{\mathcal{M}^{T}}(\mathcal{A})-\big{(}\frac{c_{A}}{2^{K}}-WM\big{)}\bigl{\rvert}\leq{\epsilon}.$$

$${\tilde{h}}^{i+1}(q,w_1{w}_2,q^{\prime })={\left[\underset{q^{\prime \prime }\in Q}{\min }\frac{{\tilde{h}}^i(q,w_1,q^{\prime \prime })+{\tilde{h}}^i(q^{\prime \prime },w_2,q^{\prime })}{2}\right]}_{{ϵ}_0}$$

$$h(q,w,q^{\prime })-{\tilde{h}}^i(q,w,q^{\prime })\le \frac{h(q,w_1,s)+h(s,w_2,q)}{2}-{\left[\frac{(a+b){ϵ}_0}{2}\right]}_{{ϵ}_0}$$

$$h(q,w,q^{\prime })-{\tilde{h}}^i(q,w,q^{\prime })\le \frac{(h(q,w_1,s)-a{ϵ}_0)+(h(s,w_2,q)-b{ϵ}_0)}{2}+\frac{{ϵ}_0}{2}$$

Full text

The exact answers to the probabilistic questions for non-deterministic automata can be irrational and are uncomputable in general. To overcome this limitation, we propose approximation algorithms for the probabilistic questions, which work in exponential time in the size of the automaton and polynomial time in the size of the Markov chain and the given precision. We apply this result to show that non-deterministic automata can be effectively determinised with respect to the standard deviation metric.

1 Introduction

Weighted automata are (non-deterministic) finite automata in which transitions carry weights [16]. We study here weighted automata (on finite and infinite words) whose semantics is given by value functions (such as the sum or the average) [11]. In such weighted automata transitions are labeled with rational numbers and hence every run yields a sequence of rationals, which the value function aggregates into a single (real) number. This number is the value of the run, and the value of a word is the infimum over the values of all accepting runs on that word.

The value function approach has been introduced to express quantitative system properties (performance, energy consumption, etc.) and it serves as a foundation for quantitative verification [11, 21]. Basic decision questions for weighted automata are quantitative counterparts of the emptiness and universality questions obtained by imposing a threshold on the values of words.

Probabilistic semantics. The emptiness and the universality problems correspond to the best-case and the worst-case analysis. For the average-case analysis, weighted automata are considered under probabilistic semantics, in which words are random events generated by a Markov chain [10, 12]. In such a setting, functions from words to reals computed by deterministic weighted automata are measurable and hence can be considered as random variables. The fundamental probabilistic questions are to compute the expected value and the cumulative distribution for a given automaton and a Markov chain.

The deterministic case. Weighted automata under probabilistic semantics have been studied only in the deterministic case. A close relationship has been established between weighted automata under probabilistic semantics and weighted Markov chains [10]. For a weighted automaton $\mathcal{A}$ and a Markov chain $\mathcal{M}$ representing the distribution over words, the probabilistic problems for $\mathcal{A}$ and $\mathcal{M}$ coincide with the probabilistic problem of the weighted Markov chain $\mathcal{A}\times\mathcal{M}$. Weighted Markov chains have been intensively studied with single and multiple quantitative objectives [4, 13, 18, 29]. The above reduction does not extend to non-deterministic weighted automata [12, Example 30].

Significance of nondeterminism. Non-deterministic weighted automata are provably more expressive than their deterministic counterpart [11]. Many important system properties can be expressed with weighted automata only in the nondeterministic setting. This includes minimal response time, minimal number of errors and the edit distance problem [21], which serves as the foundation for the specification repair framework from [6].

Non-determinism can also arise as a result of abstraction. The exact systems are often too large and complex to operate on and hence they are approximated with smaller non-deterministic models [14]. The abstraction is especially important for multi-threaded programs, where the explicit model grows exponentially with the number of threads [20].

Our contributions

We study non-deterministic weighted automata under probabilistic semantics. We work with weighted automata as defined in [11], where a value function $f$ is used to aggregate weights along a run, and the value of the word is the infimum over values of all runs. (The infimum can be changed to supremum as both definitions are dual). We primarily focus on the two most interesting value functions: the sum of weights over finite runs, and the limit average over infinite runs. The main results presented in this paper are as follows.

• •

We show that the answers to the probabilistic questions for weighted automata with the sum and limit-average value functions can be irrational and even transcendental (Theorem 3.3) and cannot be computed by any effective representation (Theorem 4.1).

• •

We establish approximation algorithms for the probabilistic questions for weighted automata with the sum and limit-average value functions. The approximation is #P-complete for (total) weighted automata with the sum value function (Theorem 5.5), and it is PSpace-hard and solvable in exponential time for weighted automata with the limit-average value function (Theorem 7.1).

• •

We show that weighted automata with the limit-average value function can be approximately determinised (Theorem 8.3). Given an automaton $\mathcal{A}$ and $\epsilon>0$, we show how to compute a deterministic automaton $\mathcal{A}_{D}$ such that the expected difference between the values returned by both automata is at most $\epsilon$.

Applications

We briefly discuss applications of our contributions in quantitative verification.

• •

The expected-value question corresponds to the average-case analysis in quantitative verification [10, 12]. Using results from this paper, we can perform the average-case analysis with respect to quantitative specifications given by non-deterministic weighted automata.

• •

Some quantitative-model-checking frameworks [11] are based on the universality problem for non-deterministic automata, which asks whether all words have the value below a given threshold. Unfortunately, the universality problem is undecidable for weighted automata with the sum or the limit average values functions. The distribution question can be considered as a computationally-attractive variant of universality, i.e., we ask whether almost all words have value below some given threshold. We show that if the threshold can be approximated, the distribution question can be computed effectively.

• •

Weighted automata have been used to formally study online algorithms [2]. Online algorithms have been modeled by deterministic weighted automata, which make choices based solely on the past, while offline algorithms have been modeled by non-deterministic weighted automata. Relating deterministic and non-deterministic models allowed for formal verification of the worst-case competitiveness ratio of online algorithms. Using the result from our paper, we can extend the analysis from [2] to the average-case competitiveness.

Related work

The problem considered in this paper is related to the following areas from the literature.

Probabilistic verification of qualitative properties. Probabilistic verification asks for the probability of the set of traces satisfying a given property. For non-weighted automata, it has been extensively studied [33, 15, 4] and implemented [26, 22]. The prevalent approach in this area is to work with deterministic automata, and apply determinisation as needed. To obtain better complexity bounds, the probabilistic verification problem has been directly studied for unambiguous Büchi automata in [5]; the authors explain there the potential pitfalls in the probabilistic analysis of non-deterministic automata.

Weighted automata under probabilistic semantics. Probabilistic verification of weighted automata and their extensions has been studied in [12]. All automata considered there are deterministic.

Markov Decision Processes (MDPs). MDPs are a classical extension of Markov chains, which models control in a stochastic environment [4, 18]. In MDPs, probabilistic and non-deterministic transitions are interleaved; this can be explained as a game between two players: Controller and Environment. Given a game objective (e.g. state reachability), the goal of Controller is to maximize the probability of the objective by selecting non-deterministic transitions. Environment is selecting probabilistic transitions at random w.r.t. a probability distribution described in the current state of the MDP. Intuitively, the non-determinism in MDPs is resolved based on the past, i.e., each time Controller selects a non-deterministic transition, its choice is based on previously picked transitions. Our setting can be also explained in such a game-theoretic framework: first, Environment generates a complete word, and only then non-deterministic choices are resolved by Controller, who generates a run of a given weighted automaton. That non-determinism in a run at some position $i$ may depend on letters in the input word on positions past $i$ (i.e., future events).

Partially Observable Markov Decision Process (POMDPs) [3] are an extension of MDPs, which models weaker non-determinism. In this setting, the state space is partitioned into observations and the non-deterministic choices have to be the same for sequences consisting of the same observations (but possibly different states). Intuitively, Controller can make choices based only on the sequence of observations it has seen so far. While in POMDPs Controller is restricted, in our setting Controller is stronger than in the MDPs case.

Non-deterministic probabilistic automata. The combination of nondeterminism with stochasticity has been recently studied in the framework of probabilistic automata [32]. There have been defined non-deterministic probabilistic automata (NPA) and there has been proposed two possible semantics for NPA. It has been shown that the equivalence problem for NPA is undecidable (under either of the considered two semantics). Related problems, such as the threshold problem, are undecidable already for (deterministic) probabilistic automata [7]. While NPA work only over finite words, the interaction between probabilistic and non-deterministic transitions is more general than in our framework. In particular, non-determinism in NPA can influence the probability distribution, which is not possible in our framework.

Approximate determinisation. As weighted automata are not determinisable, Boker and Henzinger [9] studied approximate determinisation defined as follows. The distance $d_{\sup}$ between weighted automata $\mathcal{A}_{1},\mathcal{A}_{2}$ is defined as $d_{\sup}(\mathcal{A}_{1},\mathcal{A}_{2})=\sup_{w}|\mathcal{A}_{1}(w)-\mathcal{A}_{2}(w)|$. A nondeterministic weighted automaton $\mathcal{A}$ can be approximately determinised if for every $\epsilon>0$ there exists a deterministic automaton $\mathcal{A}_{D}$ such that $d_{\sup}(\mathcal{A},\mathcal{A}_{D})\leq\epsilon$. Unfortunately, weighted automata with the limit average value function cannot be approximately determinised [9]. In this work we show that the approximate determinisation is possible for the standard deviation metric $d_{\mathrm{std}}$ defined as $d_{\mathrm{std}}(\mathcal{A}_{1},\mathcal{A}_{2})=\mathbb{E}(|\mathcal{A}_{1}(w)-\mathcal{A}_{2}(w)|)$.

This paper is an extended and corrected version of [27]. It contains full proofs, an extended discussion and a stronger version of Theorem 3.3. We have showed in [27] that the expected values and the distribution values may be irrational. In this paper we show that these values can be even transcendental (Theorem 3.3).

We have corrected two claims from [27]. First, we have corrected statements of Theorems 3.3 and 4.1. For LimAvg-automata and the distribution question $\mathbb{D}_{\mathcal{M},\mathcal{A}}(\lambda)$, the values, which can be irrational and uncomputable are not the values of the probability $\mathbb{D}_{\mathcal{M},\mathcal{A}}(\lambda)=\mathbb{P}_{\mathcal{M}}(\{w\mid\mathcal{L}_{{\mathcal{A}}}\leq\lambda\})$, but the values of the thershod $\lambda$ that correspond to mass points, i.e., values $\lambda$ such that $\mathbb{P}_{\mathcal{M}}(\{w\mid\mathcal{L}_{{\mathcal{A}}}=\lambda\})>0$. We have also removed from Theorem 5.1 PSPACE-hardness claim for the distribution question for (non-total) Sum-automata. We show that the (exact) distribution question for all Sum-automata is #P-complete.

2 Preliminaries

Given a finite alphabet $\Sigma$ of letters, a word $w$ is a finite or infinite sequence of letters. We denote the set of all finite words over $\Sigma$ by $\Sigma^{*}$, and the set of all infinite words over $\Sigma$ by $\Sigma^{\omega}$. For a word $w$, we define $w[i]$ as the $i$-th letter of $w$, and we define $w[i,j]$ as the subword $w[i]w[i+1]\ldots w[j]$ of $w$. We use the same notation for other sequences defined later on. By $|w|$ we denote the length of $w$.

A (non-deterministic) finite automaton (NFA) is a tuple $(\Sigma,Q,Q_{0},F,\delta)$ consisting of an input alphabet $\Sigma$ , a finite set of states $Q$, a set of initial states $Q_{0}\subseteq Q$, a set of final states $F$, and a finite transition relation $\delta\subseteq Q\times\Sigma\times Q$.

We define $\delta(q,a)=\{q^{\prime}\in\mathbb{Q}\mid\delta(q,a,q^{\prime})\}$ and $\delta(S,a)=\bigcup_{q\in S}\delta(q,a)$. We extend this to words $\widehat{\delta}\colon 2^{Q}\times\Sigma^{*}\to 2^{Q}$ in the following way: $\widehat{\delta}(S,\epsilon)=S$ (where $\epsilon$ is the empty word) and $\widehat{\delta}(S,aw)=\widehat{\delta}(\delta(S,a),w)$, i.e., $\widehat{\delta}(S,w)$ is the set of states reachable from $S$ via $\delta$ over the word $w$.

Weighted automata

A weighted automaton is a finite automaton whose transitions are labeled by rational numbers called weights. Formally, a weighted automaton is a tuple $(\Sigma,Q,Q_{0},F,\delta,{C})$, where the first five elements are as in the finite automata, and ${C}\colon\delta\to\mathbb{Q}$ is a function that defines weights of transitions. An example of a weighted automaton is depicted in Figure 1.

The size of a weighted automaton $\mathcal{A}$, denoted by $|\mathcal{A}|$, is $|Q|+|\delta|+\sum_{q,q^{\prime},a}\mathrm{len}(C(q,a,q^{\prime}))$, where $\mathrm{len}$ is the sum of the lengths of the binary representations of the numerator and the denominator of a given rational number.

A run $\pi$ of an automaton $\mathcal{A}$ on a word $w$ is a sequence of states $\pi[0]\pi[1]\dots$ such that $\pi[0]$ is an initial state and for each $i$ we have $(\pi[i-1],w[i],\pi[i])\in\delta$. A finite run $\pi$ of length $k$ is accepting if and only if the last state $\pi[k]$ belongs to the set of accepting states $F$. As in [11], we do not consider $\omega$-accepting conditions and assume that all infinite runs are accepting. Every run $\pi$ of an automaton $\mathcal{A}$ on a (finite or infinite) word $w$ defines a sequence of weights of successive transitions of $\mathcal{A}$ as follows. Let $({C}(\pi))[i]$ be the weight of the $i$-th transition, i.e., ${C}(\pi[i-1],w[i],\pi[i])$. Then, ${C}(\pi)=({C}(\pi)[i])_{1\leq i\leq|w|}$. A value functions $f$ is a function that assigns real numbers to sequences of rational numbers. The value $f(\pi)$ of the run $\pi$ is defined as $f({C}(\pi))$.

The value of a (non-empty) word $w$ assigned by the automaton $\mathcal{A}$, denoted by $\mathcal{L}_{{\mathcal{A}}}(w)$, is the infimum of the set of values of all accepting runs on $w$. The value of a word that has no (accepting) runs is infinite. To indicate a particular value function $f$ that defines the semantics, we will call a weighted automaton $\mathcal{A}$ an $f$-automaton.

Value functions

We consider the following value functions. For finite runs, functions Min and Max are defined in the usual manner, and the function Sum is defined as

[TABLE]

For infinite runs we consider the supremum Sup and infimum Inf functions (defined like Max and Min but on infinite runs) and the limit average function LimAvg defined as

[TABLE]

where for finite runs $\pi$ we have $\textsc{Avg}(\pi)=\frac{\textsc{Sum}(\pi)}{|C(\pi)|}$.

2.1 Probabilistic semantics

A (finite-state discrete-time) Markov chain is a tuple $\langle\Sigma,S,s_{0},E\rangle$, where $\Sigma$ is the alphabet of letters, $S$ is a finite set of states, $s_{0}$ is an initial state, $E\colon S\times\Sigma\times S\mapsto[0,1]$ is an edge probability function, which for every $s\in S$ satisfies that $\sum_{a\in\Sigma,s^{\prime}\in S}E(s,a,s^{\prime})=1$. An example of a single-state Markov chain is depicted in Figure 1.

In this paper, Markov chains serve as a mathematical model as well as the input to algorithms. Whenever a Markov chain is the input to a problem or an algorithm, we assume that all edge probabilities are rational and the size of a Markov chain $\mathcal{M}$ is defined as $|\mathcal{M}|=|S|+|E|+\sum_{q,q^{\prime},a}\mathrm{len}(E(q,a,q^{\prime}))$.

The probability of a finite word $u$ w.r.t. a Markov chain $\mathcal{M}$, denoted by $\mathbb{P}_{\mathcal{M}}(u)$, is the sum of probabilities of paths from $s_{0}$ labeled by $u$, where the probability of a path is the product of probabilities of its edges. For sets $u\cdot\Sigma^{\omega}=\{uw\mid w\in\Sigma^{\omega}\}$, called cylinders, we have $\mathbb{P}_{\mathcal{M}}(u\cdot\Sigma^{\omega})=\mathbb{P}_{\mathcal{M}}(u)$, and then the probability measure over infinite words defined by $\mathcal{M}$ is the unique extension of the above measure to the $\sigma$-algebra generated by cylinders (by Carathéodory’s extension theorem [17]) We will denote the unique probability measure defined by $\mathcal{M}$ as $\mathbb{P}_{\mathcal{M}}$. For example, for the Markov chain $\mathcal{M}$ presented in Figure 1, we have that $\mathbb{P}_{\mathcal{M}}(ab)=\frac{1}{4}$, and so $\mathbb{P}_{\mathcal{M}}(\{w\in\{a,b\}^{\omega}\mid w[0,1]=ab\})=\frac{1}{4}$, whereas $\mathbb{P}_{\mathcal{M}}(X)=0$ for any countable set of infinite words $X$.

A function $f\colon\Sigma^{\omega}\to\mathbb{R}$ the is measureable w.r.t. $\mathbb{P}_{\mathcal{M}}$ is called a random variable (w.r.t. $\mathbb{P}_{\mathcal{M}}$). A random variable $g$ is discrete, if there exists a countable set $Y\subset\mathbb{R}$ such that $g$ returns a value for $Y$ with probability $1$ ($\mathbb{P}_{\mathcal{M}}(\{w\mid g(w)\in Y\})=1$). For the discrete random variable $g$, we define the expected value $\mathbb{E}_{\mathcal{M}}(g)$ (w.r.t. the measure $\mathbb{P}_{\mathcal{M}}$) as

[TABLE]

Every non-negative random variable $h\colon\Sigma^{\omega}\to\mathbb{R}^{+}$ is a point-wise limit of some sequence of monotonically increasing discrete random variables $g_{1},g_{2},\ldots$ and the expected value $\mathbb{E}_{\mathcal{M}}(h)$ is the limit of expected values $\mathbb{E}_{\mathcal{M}}(g_{i})$ [17]. Finally, every random variable $f$ can be presented as the difference $h_{1}-h_{2}$ of non-negative random variables $h_{1},h_{2}$ and we have $\mathbb{E}_{\mathcal{M}}(f)=\mathbb{E}_{\mathcal{M}}(h_{1})-\mathbb{E}_{\mathcal{M}}(h_{2})$ [17].

A terminating Markov chain $\mathcal{M}^{T}$ is a tuple $\langle\Sigma,S,s_{0},E,T\rangle$, where $\Sigma$, $S$ and $s_{0}$ are as usual, $E\colon S\times(\Sigma\cup\{\epsilon\})\times S\mapsto[0,1]$ is the edge probability function, such that if $E(s,a,t)$, then $a=\epsilon$ if and only if $t\in T$, and for every $s\in S$ we have $\sum_{a\in\Sigma\cup\{\epsilon\},s^{\prime}\in S}E(s,a,s^{\prime})=1$, and $T$ is a set of terminating states such that the probability of reaching a terminating state from any state $s$ is positive. Notice that the only $\epsilon$-transitions in a terminating Markov chain are those that lead to a terminating state.

The probability of a finite word $u$ w.r.t. $\mathcal{M}^{T}$, denoted $\mathbb{P}_{\mathcal{M}^{T}}(u)$, is the sum of probabilities of paths from $s_{0}$ labeled by $u$ such that the only terminating state on this path is the last one. Notice that $\mathbb{P}_{\mathcal{M}^{T}}$ is a probability distribution on finite words whereas $\mathbb{P}_{\mathcal{M}}$ is not (because the sum of probabilities may exceed 1).

A function $f\colon\Sigma^{*}\to\mathbb{R}$ is called a random variable (w.r.t. $\mathbb{P}_{\mathcal{M}^{T}}$). Since words generated by $\mathcal{M}^{T}$ are finite, the co-domain of $f$ is countable and hence $f$ is discrete. The expected value of $f$ w.r.t. $\mathcal{M}^{T}$ is defined in the same way as for non-terminating Markov chains.

Automata as random variables

An infinite-word weighted automaton $\mathcal{A}$ defines the function $\mathcal{L}_{{\mathcal{A}}}$ that assigns each word from $\Sigma^{\omega}$ its value $\mathcal{L}_{{\mathcal{A}}}(w)$. This function is measurable for all the automata types we consider in this paper (see Remark 2.1 below). Thus, this function can be interpreted as a random variable with respect to the probabilistic space we consider. Hence, for a given automaton $\mathcal{A}$ (over infinite words) and a Markov chain $\mathcal{M}$, we consider the following quantities:

$\mathbb{E}_{\mathcal{M}}(\mathcal{A})$ — the expected value of the random variable $\mathcal{L}_{{\mathcal{A}}}$ w.r.t. the measure $\mathbb{P}_{\mathcal{M}}$.

$\mathbb{D}_{\mathcal{M},\mathcal{A}}(\lambda)=\mathbb{P}_{\mathcal{M}}(\{w\mid\mathcal{L}_{{\mathcal{A}}}(w)\leq\lambda\})$ — the (cumulative) distribution function of $\mathcal{L}_{{\mathcal{A}}}$ w.r.t. the measure $\mathbb{P}_{\mathcal{M}}$.

In the finite words case, the expected value $\mathbb{E}_{\mathcal{M}^{T}}$ and the distribution $\mathbb{D}_{\mathcal{M}^{T},\mathcal{A}}$ are defined in the same manner.

Remark \thetheorem (Bounds on the expected value and the distribution).

Both quantities can be easily bounded: the value of the distribution function $\mathbb{D}_{\mathcal{M},\mathcal{A}}$ is always between [math] and $1$.

For a LimAvg-automaton $\mathcal{A}$, we have $\mathbb{E}_{\mathcal{M}}(\mathcal{A})\in[\min_{\mathcal{A}},\max_{\mathcal{A}}]\cup\{\infty\}$, where $\min_{\mathcal{A}}$ and $\max_{\mathcal{A}}$ denote the minimal and the maximal weight of $\mathcal{A}$ and $\mathbb{E}_{\mathcal{M}}(\mathcal{A})=\infty$ if and only if the probability of the set of words with no accepting runs in $\mathcal{A}$ is positive. Note that we consider no $\omega$-accepting conditions, and hence all infinite runs of LimAvg-automata are accepting, but there can be infinite words, on which a given LimAvg-automaton has no infinite runs.

For a Sum-automaton $\mathcal{A}$, we have $\mathbb{E}_{\mathcal{M}^{T}}(\mathcal{A})\in[L_{\mathcal{M}^{T}}\cdot\min_{\mathcal{A}},L_{\mathcal{M}^{T}}\cdot\max_{\mathcal{A}}]\cup\{\infty\}$, where $L_{\mathcal{M}^{T}}$ is the expected length of a word generated by $\mathcal{M}^{T}$ (it can be computed in a standard way [19, Section 11.2]) and, as above, $\mathbb{E}_{\mathcal{M}^{T}}(\mathcal{A})=\infty$ if and only if there is a finite word $w$ generated by $\mathcal{M}^{T}$ with non-zero probability such that $\mathcal{A}$ has no accepting runs on $w$.

We show in Section 3.2 that the distribution and expected value may be irrational, even for integer weights and uniform distributions.

Remark \thetheorem (Measurability of functions represented by automata).

For automata on finite words, Inf-automata and Sup-automata, measurability of $\mathcal{L}_{{\mathcal{A}}}$ is straightforward. To show that $\mathcal{L}_{{\mathcal{A}}}(w)\colon\Sigma^{\omega}\mapsto\mathbb{R}$ is measurable for any non-deterministic LimAvg-automaton $\mathcal{A}$, it suffices to show that for every $x\in\mathbb{R}$, the preimage $\mathcal{L}_{{\mathcal{A}}}^{-1}(-\infty,x]$ is measurable. Let $Q$ be the set of states of $\mathcal{A}$. We can define a subset $A_{x}\subseteq\Sigma^{\omega}\times Q^{\omega}$ of the pairs, the word and the run on it, where the value of the run is less than or equal to $x$. We show that $A_{x}$ is Borel. For $p\in\mathbb{N}$, let $B_{x}^{p}$ be the subset of $\Sigma^{\omega}\times Q^{\omega}$ of pairs $(w,\pi)$ such that up to position $p$ the sequence $\pi$ is a run on $w$ and the average of weights up to $p$ is at most $x$. Observe that $B_{x}^{p}$ is an open set and $A_{x}$ is equal to $\bigcap_{\epsilon\in\mathbb{Q}^{+}}\bigcup_{p_{0}\in\mathbb{N}}\bigcap_{p\geq p_{0}}B_{x+\epsilon}^{p}$, i.e., $A_{x}$ consists of pairs $(w,\pi)$ satisfying that for every $\epsilon\in\mathbb{Q}^{+}$ there exists $p_{0}$ such that for all $p\geq p_{0}$ the average weight of $\pi$ at $p$ does not exceed $x+\epsilon$ and $\pi$ is a run on $w$ (each finite prefix is a run). Finally, $\mathcal{L}_{{\mathcal{A}}}^{-1}(-\infty,x]$ is the projection of $A_{x}$ on the first component $\Sigma^{\omega}$. The projection of a Borel set is an analytic set, which is measurable [24]. Thus, $\mathcal{L}_{{\mathcal{A}}}$ defined by a non-deterministic LimAvg-automaton is measurable.

The above proof of measurability requires some knowledge of descriptive set theory. We will give a direct proof of measurability of $\mathcal{L}_{{\mathcal{A}}}$ in the paper (Theorem 7.1).

2.2 Computational questions

We consider the following basic computational questions:

The expected value question: Given an $f$-automaton $\mathcal{A}$ and a (terminating) Markov chain $\mathcal{M}$, compute $\mathbb{E}_{\mathcal{M}}(\mathcal{A})$.

The distribution question: Given an $f$-automaton $\mathcal{A}$, a (terminating) Markov chain $\mathcal{M}$ and a threshold $\lambda\in\mathbb{Q}$, compute $\mathbb{D}_{\mathcal{M},\mathcal{A}}(\lambda)$.

Each of the above questions have its decision variant (useful for lower bounds), where instead of computing the value we ask whether the value is less than a given threshold $t$.

The above questions have their approximate variants:

The approximate expected value question: Given an $f$-automaton $\mathcal{A}$, a (terminating) Markov chain $\mathcal{M}$, $\epsilon\in\mathbb{Q}^{+}$, compute a number $y\in\mathbb{Q}$ such that $|y-\mathbb{E}_{\mathcal{M}}(\mathcal{A})|\leq\epsilon$.

The approximate distribution question: Given an $f$-automaton $\mathcal{A}$, a (terminating) Markov chain $\mathcal{M}$, a threshold $\lambda\in\mathbb{Q}$ and $\epsilon\in\mathbb{Q}^{+}$ compute a number $y\in\mathbb{Q}$ which belongs to $[\mathbb{D}_{\mathcal{M},\mathcal{A}}(\lambda-\epsilon)-\epsilon,\mathbb{D}_{\mathcal{M},\mathcal{A}}(\lambda+\epsilon)+\epsilon]$.

Remark \thetheorem.

The notion of approximation for the distribution question is based on the Skorokhod metric [8]. Let us compare here this notion with two possible alternatives: the inside approximation, where $y$ belongs to $[\mathbb{D}_{\mathcal{M},\mathcal{A}}(\lambda-\epsilon),\mathbb{D}_{\mathcal{M},\mathcal{A}}(\lambda+\epsilon)]$, and the outside approximation, where $y$ belongs to $[\mathbb{D}_{\mathcal{M},\mathcal{A}}(\lambda)-\epsilon,\mathbb{D}_{\mathcal{M},\mathcal{A}}(\lambda)+\epsilon]$.

The outside approximation is reasonable for Sum-automata, where the exact value of the probability is hard to compute, but for the LimAvg-automata its complexity is the same as computing the exact value (because the latter is difficult already for automata which return the same value for almost all words, as shown in Remark 6.9). For the inside approximation, it is the other way round: for Sum-automata it makes little sense as the problem is undecidable even for automata returning integer values, but for LimAvg-automata it is a reasonable definition as the returned values can be irrational.

We chose a definition that works for both types of automata. However, the results we present can be easily adjusted to work in the case of the outside approximation for Sum-automata and in the case of the inside approximation for the LimAvg-automata.

3 Basic properties

Consider an $f$-automaton $\mathcal{A}$, a Markov chain $\mathcal{M}$ and a set of words $X$. We denote by $\mathbb{E}_{\mathcal{M}}(\mathcal{A}\mid X)$ the expected value of $\mathcal{A}$ w.r.t. $\mathcal{M}$ restricted only to words in the set $X$ (see [17]). The following says that we can disregard a set of words with probability [math] (e.g. containing only some of the letters under uniform distribution) while computing the expected value.

Fact \thetheorem.

If $\mathbb{P}(X)=1$ then $\mathbb{E}_{\mathcal{M}}(\mathcal{A})=\mathbb{E}_{\mathcal{M}}(\mathcal{A}\mid X)$.

The proof is rather straightforward; the only interesting case is when there are some words not in $X$ with infinite values. But for all the functions we consider, one can show that in this case there is a set of words with infinite value that has a non-zero probability, and therefore $\mathbb{E}_{\mathcal{M}}(\mathcal{A})=\mathbb{E}_{\mathcal{M}}(\mathcal{A}\mid X)=\infty$.

One corollary of Fact 3 is that if $\mathcal{M}$ is, for example, uniform, then because the set $Y$ of ultimately-periodic words (i.e., words of the form $vw^{\omega}$) is countable and hence has probability [math], we have $\mathbb{E}_{\mathcal{M}}(\mathcal{A})=\mathbb{E}_{\mathcal{M}}(\mathcal{A}\mid\Sigma^{\omega}\setminus Y)$. This suggests that the values of ultimately-periodic words might not be representative for an automaton. We exemplify this in Remark 6.9, where we show an automaton whose value is irrational for almost all words, yet rational for all ultimately-periodic words.

3.1 Example of computing expected value by hand

Consider a LimAvg-automaton $\mathcal{A}$ and a Markov chain $\mathcal{M}$ depicted in Figure 1. We encourage the reader to take a moment to study this automaton and try to figure out its expected value.

The idea behind $\mathcal{A}$ is as follows. Assume that $\mathcal{A}$ is in a state $q_{l}$ for some $l\in\{a,b\}$. Then, it reads a word up to the first occurrence of a subword $ba$, where it has a possibility to go to $q_{x}$ and then to non-deterministically choose $q_{a}$ or $q_{b}$ as the next state. Since going to $q_{x}$ and back to $q_{l}$ costs the same as staying in $q_{l}$, we will assume that the automaton always goes to $q_{x}$ in such a case. When an automaton is in the state $q_{x}$ and has to read a word $w=a^{j}b^{k}$, then the average cost of a run on $w$ is $\frac{j}{j+k}$ if the run goes to $q_{b}$ and $\frac{k}{j+k}$ otherwise. So the run with the lowest value is the one that goes to $q_{a}$ if $j>k$ and $q_{b}$ otherwise.

To compute the expected value of the automaton, we focus on the set $X$ of words $w$ such that for each positive $n\in\mathbb{N}$ there are only finitely many prefixes of $w$ of the form $w^{\prime}a^{j}b^{k}$ such that $\frac{j+k}{|w^{\prime}|+j+k}\geq\frac{1}{n}$. Notice that this means that $w$ contains infinitely many $a$ and infinitely many $b$. It can be proved in a standard manner that $\mathbb{P}_{\mathcal{M}}(X)=1$.

Let $w\in X$ be a random event, which is a word generated by $\mathcal{M}$. Since $w$ contains infinitely many letters $a$ and $b$, it can be partitioned in the following way. Let $w=w_{1}w_{2}w_{3}\dots$ be a partition of $w$ such that each $w_{i}$ for $i>0$ is of the form $a^{j}b^{k}$ for $j\geq 0,k>0$, and for $i>1$ we also have $j>0$. For example, the partition of $w=baaabbbaabbbaba\dots$ is such that $w_{1}=b$, $w_{2}=aaabbb$, $w_{3}=aabbb$, $w_{4}=ab$, …. Let $s_{i}=|w_{1}w_{2}\dots w_{i}|$.

We now define a run $\pi_{w}$ on $w$ as follows:

[TABLE]

where the length of each block of $q_{i}$ is $|w_{i}|-1$, $q^{w}_{0}=q_{a}$ and $q^{w}_{i}=q_{a}$ if $w_{i}=a^{j}b^{k}$ for some $j>k$ and $q^{w}_{i}$=$q_{b}$ otherwise. It can be shown by a careful consideration of all possible runs that this run’s value is the infimum of values of all the runs on this word.

Lemma 3.1**.**

For every $w\in X$ we have $\mathcal{L}_{{\mathcal{A}}}(w)=\textsc{LimAvg}(\pi_{w})$.

Proof 3.2**.**

We show that for every accepting run $\pi$ on $w\in X$ we have $\textsc{LimAvg}(\pi_{w})\leq\textsc{LimAvg}(\pi)$. It follows that $\mathcal{L}_{{\mathcal{A}}}(w)=\textsc{LimAvg}(\pi_{w})$.

Consider a run $\pi$ of $\mathcal{A}$ on $w$. The cost of a run over $w_{i}=a^{j}b^{k}$ is at least $min(j,k)-1$, which is reached by $\pi_{w}$, therefore for every $i\in\mathbb{N}$ we have

[TABLE]

It may happen, however, that for some $p$, the value of $\textsc{Avg}(\pi[0,p])$ is less than $\textsc{Avg}(\pi_{w}[0,p])$; for example, for a word starting with $baaabbbb$, we have $\pi_{w}[0,4]=q_{a}q_{x}q_{b}q_{b}q_{b}$ and $\textsc{Avg}(\pi_{w}[0,4])$ is $\frac{1}{2}$, but for a run $\pi^{\prime}=q_{a}q_{x}q_{a}q_{a}q_{a}\dots$ we have $\textsc{Avg}(\pi^{\prime}[0,4])=0$. For arbitrary words, a run that never visits $q_{b}$ may have a better value. We show, however, that for words from $X$ this is not the case.

We show that for any position $p$ such that $s_{i}<p<s_{i+1}$,

[TABLE]

Observe that

[TABLE]

By (1) and the fact that the weights of the automaton do not exceed 1, we obtain

[TABLE]

thus (2).

Assume $n\in\mathbb{N}$. By the definition of $X$, there can be only finitely many prefixes of $w$ of the form $w^{\prime}a^{j}b^{k}$ where $\frac{j+k}{|w^{\prime}|+j+k}\geq\frac{1}{n}$, so $\textsc{Avg}(\pi_{w}[0,p])\geq\textsc{Avg}(\pi[0,s_{i}])+\frac{1}{n}$ may hold only for finitely many $p$. Therefore, $\textsc{LimAvg}(\pi_{w})\leq\textsc{LimAvg}(\pi)+\frac{1}{n}$ for every $n$, so $\textsc{LimAvg}(\pi_{w})\leq\textsc{LimAvg}(\pi)$.

By Fact 3 and Lemma 3.1, it remains to compute the expected value of $\textsc{LimAvg}(\{\pi_{w}\mid w\in X\})$. As the expected value of the sum is the sum of expected values, we can state that

[TABLE]

It remains to compute $\mathbb{E}_{\mathcal{M}}(({C}(\pi_{w}))[i])$. If $i$ is large enough (and since the expected value does not depend on a finite number of values, we assume that it is), the letter $\pi_{w}[i]$ is in some block $w_{s}=a^{j}b^{k}$. There are $j+k$ possible letters in this block, and the probability that the letter $\pi_{w}[i]$ is an $i$th letter in such a block is $2^{-(j+k+2)}$ (“+2”, because the block has to be maximal, so we need to include the letters before the block and after the block). So the probability that a letter is in a block $a^{j}b^{k}$ is $\frac{j+k}{2^{j+k+2}}$. The average cost of a such a letter is $\frac{\min(j,k)}{j+k}$, as there are $j+k$ letters in this block and the block contributes $\min(j,k)$ to the sum.

It can be analytically checked that

[TABLE]

We can conclude that $\mathbb{E}_{\mathcal{M}}(\textsc{LimAvg}(\pi_{w}))=\frac{1}{3}$ and, by Lemma 3.1, $\mathbb{E}_{\mathcal{M}}(\mathcal{A})=\frac{1}{3}$.

The bottom line is that even for such a simple automaton with only one strongly connected component consisting of three states (and two of them being symmetrical), the analysis is complicated. On the other hand, we conducted a simple Monte Carlo experiment in which we computed the value of this automaton on 10000 random words of length $2^{22}$ generated by $\mathcal{M}$, and observed that the obtained values are in the interval $[0.3283,0.3382]$, with the average of $0.33336$, which is a good approximation of the expected value $0.(3)$. This foreshadows our results for LimAvg-automata: we show that computing the expected value is, in general, impossible, but it is possible to approximate it with arbitrary precision. Furthermore, the small variation of the results is not accidental – we show that for strongly-connected LimAvg-automata, almost all words have the same value (which is equal to the expected value).

3.2 Irrationality of the distribution and the expected value

We show that the exact values in the probabilistic questions for Sum-automata and LimAvg-automata may be (strongly) irrational. More precisely, we show that for the Sum-automaton depicted in Figure 2, the distribution $\mathbb{D}_{\mathcal{A}}(-1)$ is transcendental, i.e., it is irrational and, unlike for instance $\sqrt{2}$, there is no polynomial with integer coefficients whose one of the roots is $\mathbb{D}_{\mathcal{A}}(-1)$. For the expected value, we construct an automaton $\mathcal{A}^{\prime}$ such that $\mathbb{E}(\mathcal{A})-\mathbb{E}(\mathcal{A}^{\prime})=1-\mathbb{D}_{\mathcal{A}}(-1)$ is transcendental. Therefore, one of $\mathbb{E}(\mathcal{A})$, $\mathbb{E}(\mathcal{A}^{\prime})$ is transcendental. Furthermore, we modify $\mathcal{A}$ and $\mathcal{A}^{\prime}$ to show that there exists LimAvg-automaton $\mathcal{A}_{\infty}$ whose expected value is transcendental and the value $\lambda$ such that is transcendental. It follows that the minimal $\lambda$ such that $\mathbb{D}_{\mathcal{A}_{\infty}}(\lambda)=1$ is transcendental.

Theorem 3.3** (Irrational values).**

The following conditions hold:

There exists a Sum-automaton whose distribution and expected value w.r.t. the uniform distribution are transcendental. 2. 2.

There exists a LimAvg-automaton such that the expected value and the value of almost all words w.r.t. the uniform distribution are transcendental.

Proof 3.4**.**

We assume that the distribution of words is uniform. In the infinite case, this means that the Markov chain contains a single state where it loops over any letter with probability $\frac{1}{|\Sigma|}$, where $\Sigma$ is the alphabet. In the finite case, this amounts to a terminating Markov chain with one regular state and one terminating state; it loops over any letter in the non-terminating state with probability $\frac{1}{|\Sigma|+1}$ or go to the terminating state over $\epsilon$ with probability $\frac{1}{|\Sigma|+1}$. Below we omit the Markov chain as it is fixed (for a given alphabet).

We define a Sum-automaton $\mathcal{A}$ (Figure 2) over the alphabet $\Sigma=\{a,\#\}$ such that $\mathcal{A}(w)=0$ if $w=a\#a^{4}\#\ldots\#a^{4^{n}}$ and $\mathcal{A}(w)\leq-1$ otherwise. Such an automaton basically picks a block with an inconsistency and verifies it. For example, if $w$ contains a block $\#a^{i}\#a^{j}\#$, the automaton $\mathcal{A}$ first assigns $-4$ to each letter $a$ and upon $\#$ it switches to the mode in which it assigns $1$ to each letter $a$. Then, $\mathcal{A}$ returns the value $j-4\cdot i$. Similarly, we can encode the run that returns the value $4\cdot i-j$. Therefore, all the runs return [math] if and only if each block of $a$’s is four times as long as the previous block. Finally, $\mathcal{A}$ checks whether the first block of $a$’s has length $1$ and returns $-1$ otherwise.

Let $\gamma$ be the probability that a word is of the form $a\#a^{4}\#\ldots\#a^{4^{n}}$. Such a word has length $l_{n}=\frac{4^{n+1}-1}{3}+n$ and its probability is ${3^{-(l_{n}+1)}}$ (as the probability of any given word with $m$ letters over a two-letters alphabet is $3^{-(m+1)}$). Therefore $\gamma$ is equal to $\sum_{n=0}^{\infty}{3^{-(l_{n}+1)}}$. Observe that $\gamma$ written in base $3$ has arbitrary long sequences of [math]’s and hence its representation is acyclic. Thus, $\gamma$ is irrational.

Due to Roth’s Theorem [30], if $\alpha\in\mathbb{R}$ is algebraic but irrational, then there are only finitely many pairs $(p,q)$ such that $|\alpha-\frac{p}{q}|\leq\frac{1}{q^{3}}$. We show that there are infinitely many such pairs for $\gamma$ and hence it is transcendental. Consider $i\in\mathbb{N}$ and let $p_{i},q_{i}\in\mathbb{N}$ be such that $q_{i}=3^{-(l_{i}+1)}$ and $\frac{p_{i}}{q_{i}}=\sum_{n=0}^{i}{3^{-(l_{n}+1)}}$. Then,

[TABLE]

Observe that for $i>1$ we have $l_{i+1}>3(l_{i}+1)$ and hence

[TABLE]

Therefore, $\gamma$ is transcendental.

Observe that $\gamma=1-\mathbb{D}_{\mathcal{A}}(-1)$. Therefore, $\mathbb{D}_{\mathcal{A}}(-1)$ is transcendental. For the expected value, we construct $\mathcal{A}^{\prime}$ such that for every word $w$ we have $\mathcal{L}_{{\mathcal{A}^{\prime}}}(w)=min(\mathcal{L}_{{\mathcal{A}}}(w),-1)$. This can be done by adding to $\mathcal{A}$ an additional initial state $q_{0}$, which starts an automaton that assigns to all words value $-1$. Observe that $\mathcal{A}$ and $\mathcal{A}^{\prime}$ differ only on words $w$ of the form $a\#a^{4}\#\ldots\#a^{4^{n}}$, where $\mathcal{A}(w)=0$ and $\mathcal{A}^{\prime}(w)=-1$. On all other words, both automata return the same values. Therefore, $\mathbb{E}(\mathcal{A})-\mathbb{E}(\mathcal{A}^{\prime})=\gamma$. It follows that at least one of the values $\mathbb{E}(\mathcal{A})$, $\mathbb{E}(\mathcal{A}^{\prime})$ is transcendental.

The same construction works for LimAvg-automata. We take $\mathcal{A}$ defined as above and convert it to a LimAvg-automaton $\mathcal{A}_{\infty}$ over \Sigma^{\prime}=\Sigma\cup\{\}$, where the fresh letter$$$resets the automaton, i.e.,$\mathcal{A}{\infty}$has transitions labeled by$$$from any final state of$\mathcal{A}$to any of its initial states. We apply the same construction to$\mathcal{A}^{\prime}$defined as above and denote the resulting automaton by$\mathcal{A}{\infty}^{\prime}$. Observe that$\mathbb{E}(\mathcal{A}{\infty})=\mathbb{E}(\mathcal{A})$(resp.,$\mathbb{E}(\mathcal{A}{\infty}^{\prime})=\mathbb{E}(\mathcal{A}^{\prime})$. To see that, consider random variables$X_{1},X_{2},\ldots$defined on$\Sigma^{\omega}$, where$X_{i}(w)$is the average value$\frac{1}{|u|}\mathcal{L}{{\mathcal{A}}}(u)$of the$i$-th block$$u$$in$w$, i.e.,$w=u{1}$u_{2}$\ldots$u_{i}$\ldots$, all$u_{j}$are from$\Sigma^{}$and$u=u_{i}$. Observe that$X_{1},X_{2},\ldots$are independent and identically distributed random variables and hence with probability$1$ we have*

[TABLE]

Therefore, with probability $1$ over words $w$ we have $\mathcal{L}_{{\mathcal{A}_{\infty}}}(w)=\mathbb{E}(\mathcal{A})$. It follows that $\mathbb{E}(\mathcal{A}_{\infty})=\mathbb{E}(\mathcal{A})$ and the minimal $\lambda$ such that $\mathbb{D}_{\mathcal{A}_{\infty}}(\lambda)=1$ equals $\mathbb{E}(\mathcal{A})$. Similarly, $\mathbb{E}(\mathcal{A}_{\infty}^{\prime})=\mathbb{E}(\mathcal{A}^{\prime})$ and the minimal $\lambda$ such that $\mathbb{D}_{\mathcal{A}_{\infty}^{\prime}}(\lambda)=1$ equals $\mathbb{E}(\mathcal{A}^{\prime})$. Therefore, for one of automata $\mathcal{A}_{\infty},\mathcal{A}_{\infty}^{\prime}$, the value of almost all words and the expected value are transcendental.

4 The exact value problems

In this section we consider the probabilistic questions for non-deterministic Sum-automata and LimAvg-automata, i.e., the problems of computing the exact values of the expected value $\mathbb{E}_{\mathcal{M}}(\mathcal{A})$ and the distribution $\mathbb{D}_{\mathcal{M},\mathcal{A}}(\lambda)$ w.r.t. a Markov chain $\mathcal{M}$ and an $f$-automaton $\mathcal{A}$. The answers to these problems are related values may be irrational (Theorem 3.3), but one can perhaps argue that there might be some representation of irrational numbers that can be employed to avoid this problem. We prove that this is not the case by showing that computing the exact value to any representation with decidable equality of two numbers is impossible.

Theorem 4.1**.**

The following conditions hold:

The expected value and the distribution of (non-deterministic) Sum-automata are uncomputable even for the uniform probability measure. 2. 2.

The expected value and the value of almost all words (if it exists) of (non-deterministic) LimAvg-automata are uncomputable even for the uniform probability measure.

Proof 4.2**.**

The proof is by a (Turing) reduction from the quantitative universality problem for Sum-automata, which is undecidable [25, 1]:

The quantitative universality problem for Sum-automata*: Given a Sum-automaton with weights $-1$, [math] and $1$, decide whether for all words $w$ we have $\mathcal{L}_{{\mathcal{A}}}(w)\leq 0$.*

We first discuss reductions to the probabilistic problems for Sum-automata. Consider an instance of the quantitative universality problem, which is a Sum-automaton $\mathcal{A}$. If there is a word $w$ with the value greater than [math], then due to uniformity of the probability measure we have , and thus $\mathbb{D}_{\mathcal{A}}(0)<1$. Otherwise, clearly $\mathbb{D}_{\mathcal{A}}(0)=1$. Therefore, solving the universality problem amounts to computing whether the $\mathbb{D}_{\mathcal{A}}(0)=1$, and thus the latter problem is undecidable. For the expected value, we construct a Sum-automaton $\mathcal{A}^{\prime}$ such that for every word $w$ we have $\mathcal{L}_{{\mathcal{A}^{\prime}}}(w)=min(\mathcal{L}_{{\mathcal{A}}}(w),0)$. Observe that $\mathbb{E}(\mathcal{A})=\mathbb{E}(\mathcal{A}^{\prime})$ if and only if for every word $w$ we have $\mathcal{L}_{{\mathcal{A}}}(w)\leq 0$, i.e., the answer to the universality problem is YES. Therefore, there is no Turing machine, which given a Sum-automaton $\mathcal{A}$ computes $\mathbb{E}(\mathcal{A})$ (in any representation allowing for effective equality testing).

For the LimAvg case, we construct a LimAvg-automaton $\mathcal{A}_{\infty}$ from the Sum-automaton $\mathcal{A}$, by connecting all accepting states (of $\mathcal{A}$) with all initial states by transitions of weight [math] labeled by an auxiliary letter $\#$. We construct $\mathcal{A}_{\infty}^{\prime}$ from $\mathcal{A}^{\prime}$ in the same way. The automata $\mathcal{A}_{\infty},\mathcal{A}_{\infty}^{\prime}$ have been constructed from $\mathcal{A}$ and respectively $\mathcal{A}^{\prime}$ as in the proof of Theorem 3.3, and the virtually the same argument shows that for almost all words $w$ (i.e., with the probability $1$) we have $\mathcal{L}_{{\mathcal{A}_{\infty}}}(w)=\mathbb{E}{\mathcal{A}}$ (resp., $\mathcal{L}_{{\mathcal{A}_{\infty}^{\prime}}}(w)=\mathbb{E}(\mathcal{A}^{\prime})$). Therefore, $\mathbb{E}(\mathcal{A}_{\infty})=\mathbb{E}(\mathcal{A}_{\infty}^{\prime})$ if and only if for every finite word $u$ we have $\mathcal{L}_{{\mathcal{A}}}(u)\leq 0$. In consequence, there is no Turing machine computing the expected value of a given LimAvg-automaton. Furthermore, since $\mathcal{A}_{\infty}$ (resp., $\mathcal{A}_{\infty}^{\prime}$) returns $\mathbb{E}(\mathcal{A}_{\infty})$ (resp., $\mathbb{E}(\mathcal{A}_{\infty}^{\prime})$) on almost all words, there is no Turing machine computing the value of almost all words of a given (non-deterministic) LimAvg-automaton.

4.1 Extrema automata

We discuss the distribution problem for Min-, Max-, Inf- and Sup-automata, where Min and Max return the minimal and respectively the maximal element of a finite sequence, and Inf and Sup return the minimal and respectively the maximal element of an infinite sequence. The expected value of an automaton can be easily computed based on the distribution as there are only finitely many possible values of a run (each possible value is a label of some transition).

Theorem 4.3**.**

For Min-, Max-, Inf- and Sup-automata $\mathcal{A}$ and a Markov chain $\mathcal{M}$, the expected value and the distribution problems can be solved in exponential time in $|\mathcal{A}|$ and polynomial time in $|\mathcal{M}|$.

Proof 4.4**.**

We discuss the case of $f=\textsc{Inf}$ as the other cases are similar. Consider an Inf-automaton $\mathcal{A}$. Observe that every value returned by $\mathcal{A}$ is one of its weights. For each weight $x$ of $\mathcal{A}$, we construct a (non-deterministic) $\omega$-automaton $\mathcal{A}_{x}$ that accepts only words of value greater than $x$, i.e., $\mathcal{L}_{{\mathcal{A}_{x}}}=\{w\mid\mathcal{L}_{{\mathcal{A}}}(w)>x\}$. To construct $\mathcal{A}_{x}$, we take $\mathcal{A}$, remove the transitions of weight less or equal to $x$, and drop all the weights. Therefore, the set of words with the value greater than $x$ is regular, and hence it is measurable and we can compute its probability $p_{x}$ by computing the probability of $\mathcal{L}_{{\mathcal{A}_{x}}}$. The probability of an $\omega$-regular language given by a non-deterministic $\omega$-automaton (without acceptance conditions) can be computed in exponential time in the size of the automaton and polynomial time in the Markov chain defining the probability distribution [4, Chapter 10.3]. It follows that $p_{x}$ can be computed in exponential time in $|\mathcal{A}|$ and polynomial time in $|\mathcal{M}|$.

Observe that $p_{x}=1-\mathbb{D}_{\mathcal{M},\mathcal{A}}(x)$ and hence we can compute the distribution question $\mathbb{D}_{\mathcal{M},\mathcal{A}}(\lambda)$ by computing $1-p_{x}$ for the maximal weight $x$ that does not exceed $\lambda$. For the expected value, let $x_{1},\ldots,x_{k}$ be all weights of $\mathcal{A}$ listed in the ascending order. Let $p_{0}=1$. Observe that for all $i\in\{1,\ldots,k\}$ we have $p_{x_{i-1}}-p_{x_{i}}=$ is the probability of the set of words of the value $x_{i}$. Therefore, $\mathbb{E}_{\mathcal{M}}(\mathcal{A})=\sum_{i=1}^{k}(p_{x_{i-1}}-p_{x_{i}})\cdot x_{i}$ and hence the expected value can be computed in exponential time in $|\mathcal{A}|$ and polynomial time in $|\mathcal{M}|$.

5 The approximation problems

We start the discussion on the approximation problems by showing a hardness result that holds for a wide range of value functions. We say that a function is [math]-preserving if its value is [math] whenever the input consists only of [math]s. The functions Sum, LimAvg, Min, Max, Inf, Sup and virtually all the functions from the literature [11] are [math]-preserving. The hardness result follows from the fact that accepted words have finite values, which we can force to be [math], while words without accepting runs have infinite values.

The answers in the approximation problems are numbers and to study the lower bounds, we consider their decision variants, called the separation problems. The expected separation problem is a variant of the expected value problem, in which the input is enriched with numbers $a,b$ such that $b-a>2\epsilon$ and the instance is such that $\mathbb{E}_{\mathcal{M}}(\mathcal{A})\not\in[a,b]$ and the question is whether $\mathbb{E}_{\mathcal{M}}(\mathcal{A})<a$. In the distribution separation problem, the input is enriched with numbers $a,b,c,d$ such that $b-a>2\epsilon$ and $d-c>2\epsilon$, the instance satisfies for all $\lambda\in[c,d]$ we have $\mathbb{D}_{\mathcal{M},\mathcal{A}}(\lambda)\not\in[a,b]$, and we ask whether $\mathbb{D}_{\mathcal{M},\mathcal{A}}(\frac{c+d}{2})<a$. Note that having an algorithm computing one of the approximate problems (for the distribution or the expected value), we can use it to decide the separation question. Conversely, using the separation problem as an oracle, we can perform binary search on the domain to solve the corresponding approximation problem in polynomial time.

Theorem 5.1**.**

The following conditions hold:

For any [math]-preserving function $f$, the expected separation problem for non-deterministic $f$-automata is PSpace-hard. 2. 2.

For any [math]-preserving function $f$ over infinite words, the distribution separation problem for non-deterministic $f$-automata over infinite words is PSpace-hard.

Proof 5.2**.**

The proof is via reduction from the universality question for non-deterministic (unweighted) finite-word automata, which is PSpace-complete [23].

The finite-word case*. We consider the uniform distribution over finite words. Given a non-deterministic finite-word automaton $\mathcal{A}$, we construct a finite-word $f$-automaton $\mathcal{A}_{\mathrm{fin}}$ by labeling all transitions of $\mathcal{A}$ with [math]. Observe that if there exists a word which is not accepted by $\mathcal{A}$ then the expected value of $\mathcal{A}_{\mathrm{fin}}$ is $\infty$. Otherwise, all words have value [math] and hence the expected value for $\mathcal{A}_{\mathrm{fin}}$ is [math]. The universality problem for $\mathcal{A}$ reduces to the expected separation problem for $\mathcal{A}_{\mathrm{fin}}$.*

The infinite-word case*. We consider the uniform distribution over infinite words. Given a non-deterministic finite-word automaton $\mathcal{A}$, we construct an infinite-word $f$-automaton $\mathcal{A}_{\infty}$ in the following way. We start with the automaton $\mathcal{A}$. First, we extend the input alphabet with a fresh letter $\#$, which resets the automaton. More precisely, we add transitions labeled by $\#$ between any final state of $\mathcal{A}$ and any initial state of $\mathcal{A}$. Finally, we label all transitions with [math]. The resulting automaton is $\mathcal{A}_{\infty}$.*

If there exists a finite word $u$ rejected by $\mathcal{A}$, then for every infinite word $w$ containing the infix $\#u\#$ the automaton $\mathcal{A}_{\infty}$ has no infinite run and hence it assigns value $\infty$ to $w$. Observer that the set of words containing $\#u\#$ has probability $1$ (for any finite word $u$). Therefore, if $\mathcal{A}$ rejects some word, the expected value for $\mathcal{A}_{\infty}$ is $\infty$ and the distribution of $\mathcal{A}_{\infty}$ for any $\lambda\in\mathbb{R}$ is [math]. Otherwise, if $\mathcal{A}$ accepts all words, the expected value of $\mathcal{A}_{\infty}$ is [math] and the distribution of $\mathcal{A}_{\infty}$ for any $\lambda\geq 0$ is $1$. The universality problem for $\mathcal{A}$ reduces to the separation problems for $\mathcal{A}_{\infty}$.

Total automata

Theorem 5.1 gives us a general hardness result, which is due to accepting conditions rather than values returned by weighted automata. In the following, we focus on weights and we assume that weighted automata are total, i.e., they accept all words (resp., almost all words in the infinite-word case). For Sum-automata under the totality assumption, the approximate probabilistic questions become #P-complete. We additionally show that the approximate distribution question for Sum-automata is in #P regardless of totality assumption.

Theorem 5.3**.**

The following conditions hold:

The approximate expected value and the approximate distribution questions for total non-deterministic total Sum-automata are #P-complete. 2. 2.

The approximate distribution question for non-deterministic Sum-automata is #P-complete.

Proof 5.4**.**

#P-hardness*. Consider the problem of counting the number of satisfying assignment of a given propositional formula $\varphi$ in Conjunctive Normal Form (CNF) [31, 28]. This problem is #P-complete. We reduce it to the problem of approximation of the expected value for total Sum-automata. Consider a formula $\varphi$ in CNF over $n$ variables. Let $\mathcal{M}^{T}$ be a terminating Markov chain over $\{0,1\}$, which at each step produces [math] and $1$ with equal probability $\frac{1}{3}$, and it terminates with probability $\frac{1}{3}$. We define a total Sum-automaton $\mathcal{A}_{\varphi}$ such that it assigns [math] to all words of length different than $n$. For words $u\in\{0,1\}^{n}$, the automaton $\mathcal{A}_{\varphi}$ regards $u$ as an assignment for variables of $\varphi$; $\mathcal{A}_{\varphi}$ non-deterministically picks one clause of $\varphi$ and returns $1$ if that clause is satisfied and [math] otherwise. We can construct such $\mathcal{A}_{\varphi}$ to have polynomial size in $|\varphi|$. Observe that $\mathcal{A}_{\varphi}(u)=0$ if some clause of $\varphi$ is not satisfied by $u$, i.e., $\varphi$ is false under the assignment given by $u$. Otherwise, if the assignment given by $u$ satisfies $\varphi$, then $\mathcal{A}_{\varphi}(u)=1$. It follows that the expected value of $\mathcal{A}_{\varphi}$ equals ${3}^{-(n+1)}\cdot C$, where ${3}^{-(n+1)}$ is the probability of generating a word of length $n$ and $C$ is the number of variable assignments satisfying $\varphi$. Therefore, we can compute $C$ by computing the expected value of $\mathcal{A}_{\varphi}$ with any $\epsilon$ less than $0.5\cdot{3}^{-(n+1)}$. Observe that the automaton $\mathcal{A}_{\varphi}$ returns values [math] and $1$ and hence the expected value $\mathbb{E}_{\mathcal{M}}(\mathcal{A}_{\varphi})=1-\mathbb{D}_{\mathcal{M},\mathcal{A}_{\varphi}}(0)$, where $1-\mathbb{D}_{\mathcal{M},\mathcal{A}_{\varphi}}(0)$ is the probability that $\mathcal{A}_{\varphi}$ returns $1$.*

Containment of the approximate distribution question in #P*. Consider a terminating Markov chain $\mathcal{M}^{T}$, a Sum-automaton $\mathcal{A}$, and $\epsilon\in\mathbb{Q}^{+}$.*

Let $C$ be the smallest number such that every non-zero probability in $\mathcal{M}^{T}$ is at least $2^{-C}$. Such $C$ is polynomial in the input size. Consider $N=C\cdot\mathrm{len}(\epsilon)+1$ and let $\mathbb{D}_{\mathcal{M}^{T},\mathcal{A}}(\lambda,N)$ be the distribution of $\mathcal{A}$ over words up to length $N$, i.e., . We show that the distribution of $\mathcal{A}$ and the distribution of $\mathcal{A}$ over words up to length $N$ differ by less than $\frac{\epsilon}{2}$, i.e., that

[TABLE]

To do so, let $p_{n}$, for $n\in\mathbb{N}$, be the probability that $\mathcal{M}^{T}$ emits a word of the length greater than $n$. From any state of $\mathcal{M}^{T}$, the probability of moving to a terminating state is at least $2^{-C}$. We can (very roughly) bound the probability of generating a word of length greater than $i$ ($p_{i})$ by $(1-2^{-C})^{i}$. This means that $p_{n}$ decreases exponentially with $n$. Since $(1-\frac{1}{n})^{n}\leq\frac{1}{2}$ for all $n>1$, we obtained the desired inequality.

Let $K=(N+1)\cdot\log(|\Sigma|)\cdot\epsilon^{-1}+1$. We build a non-deterministic Turing machine $H_{1}$ such that on the input $\mathcal{M}^{T}$, $\mathcal{A}$, $\epsilon$, and $\lambda$ such that the number $c_{A}$ of accepting computations of $H_{1}$ satisfies the following:

[TABLE]

To $\epsilon$-approximate $\mathbb{D}_{\mathcal{M}^{T},\mathcal{A}}(\lambda)$, we need to compute $c_{A}$ and divide it by $2^{K}$, which can be done in polynomial time.

The machine $H_{1}$ works as follows. Given the input $\mathcal{M}^{T}$, $\mathcal{A}$, $\epsilon$, it non-deterministically generates a string $u\alpha$, where $u\in(\Sigma\cup\{\#\})^{N}$ is a word and $\alpha\in\{0,1\}^{K}$ is a number written in binary. The machine rejects unless $u$ is of the form $wv$, where $w\in\Sigma^{*}$ and $v\in\{\#\}^{*}$. Then, the machine accepts if $\mathcal{L}_{{\mathcal{A}}}(w)\leq\lambda$ and $\alpha\leq 2^{K}\cdot$. Therefore, provided that $H_{1}$ generates $w$ with $\mathcal{L}_{{\mathcal{A}}}(w)\leq\lambda$, the number of accepting computations $c_{A}^{w}$ equals $\lfloor 2^{K}\cdot$. It follows that $c_{A}^{w}$ divided by $2^{K}$ is a $2^{-K}$-approximation of , i.e.,

[TABLE]

The total number of accepting paths of $H_{1}$ is given by

[TABLE]

We estimate the difference between $\mathbb{D}_{\mathcal{M}^{T},\mathcal{A}}(\lambda,n)$ and the value $\frac{c_{A}}{2^{K}}$:

[TABLE]

Containment of the approximate expected value question in #P*

Assume that $\mathcal{A}$ is total. For readability we assume that $\mathcal{A}$ has only integer weights. If it does not, we can multiply all weights by least common multiple of all denominators of weights in $\mathcal{A}$; this operation multiples the expected value by the same factor.*

Recall that $C$ is the smallest number such that every non-zero probability in $\mathcal{M}^{T}$ is at least $2^{-C}$. Let $W$ be the maximal absolute value of weights in $\mathcal{A}$ and let $M=C\mathrm{len}(\epsilon)\cdot\log(CW)+1$ and $\mathbb{E}_{\mathcal{M}^{T}}(\mathcal{A},N)$ be the expected value of $\mathcal{M}^{T}$ for words up to length $M$, i.e., computing only the finite sum from the definition of the expected value. We show that

[TABLE]

Recall that $p_{n}$ is the probability that $\mathcal{M}^{T}$ emits a word of the length greater than $n$, and $p_{n}\leq(1-2^{-C})^{n}$. Since $\mathcal{A}$ is total, the value of every word $w$ is finite and it belongs to the interval $[-|w|\cdot W,|w|\cdot W]$. The value of a word of the length bounded by $i$ is at most $i\cdot W$. Therefore, the expected value of $\mathcal{A}$ over words of grater than $k$ is bounded from above by $\sum_{i\geq k}p_{i}\cdot i\cdot W\leq W\cdot(1-2^{-C})^{k}\cdot(k+1)$.

W.l.o.g. we assume, that there are no transitions to the initial state in $\mathcal{A}$. Next, we transform $\mathcal{A}$ to an automaton $\mathcal{A}^{\prime}$ that returns natural numbers on all words of length at most $M$ by adding $W\cdot M$ to every transition from the initial state. Observe that $\mathbb{E}_{\mathcal{A}^{\prime}}=\mathbb{E}_{\mathcal{A}}+W\cdot M$ and $D=2\cdot W\cdot M$ is an upper bound on values returned by the automaton $\mathcal{A}^{\prime}$ on words of length at most $M$.

Finally, we construct a Turing machine $H_{2}$, similar to $H_{1}$. Let $K=(D+1)\cdot(N+1)\cdot(|\Sigma|+1)\cdot\epsilon^{-1}+1$. $H_{2}$ non-deterministically chooses a word $u\alpha$, where $u\in(\Sigma\cup\{\#\})^{N}$ is a word and $\alpha\in\{0,1\}^{K}$ is a number written in binary, and also non-deterministically picks a natural number $\beta\in[0,D]$. The machine rejects unless $u$ is of the form $wv$, where $w\in\Sigma^{*}$ and $v\in\{\#\}^{*}$. Then $H_{2}$ accepts if and only if $\mathcal{L}_{{\mathcal{A}^{\prime}}}(w)\leq\beta$ and $\alpha\leq 2^{K}\cdot$. Then, provided that $H_{2}$ generates $w$, the number of accepting computations $c_{A}^{w}$ equals $\lfloor 2^{K}\cdot$. Therefore, using estimates similar to the distribution case, we obtain the desired inequality

[TABLE]

Finally, we obtain that $\frac{c_{A}}{2^{K}}-WM$ is an $\epsilon$-approximation of $\mathbb{E}_{\mathcal{M}^{T}}(\mathcal{A})$, i.e.,

[TABLE]

We show that the approximation problem for LimAvg-automata is PSpace-hard over the class of total automata.

Theorem 5.5**.**

The separation problems for non-deterministic total LimAvg-automata are PSpace-hard.

Proof 5.6**.**

We consider the uniform distribution over infinite words. Given a non-deterministic finite-word automaton $\mathcal{A}$, we construct an infinite-word LimAvg-automaton $\mathcal{A}_{\infty}$ from $\mathcal{A}$ in the following way. We introduce an auxiliary symbol $\#$ and we add transitions labeled by $\#$ between any final state of $\mathcal{A}$ and any initial state of $\mathcal{A}$. Then, we label all transitions of $\mathcal{A}_{\infty}$ with [math]. Finally, we connect all non-accepting states of $\mathcal{A}$ with an auxiliary state $q_{\mathrm{sink}}$, which is a sink state with all transitions of weight $1$. The automaton $\mathcal{A}_{\infty}$ is total.

Observe that if $\mathcal{A}$ is universal, then $\mathcal{A}_{\infty}$ has a run of value [math] on every word. Otherwise, if $\mathcal{A}$ rejects a word $w$, then upon reading a subword $\#w\#$, the automaton $\mathcal{A}_{\infty}$ reaches $q_{\mathrm{sink}}$, i.e., the value of the whole word is $1$. Almost all words contain an infix $\#w\#$ and hence almost all words have value $1$. Therefore, the universality problem for $\mathcal{A}$ reduces to the problem deciding whether for almost all words $w$ we have $\mathcal{L}_{{\mathcal{A}_{\infty}}}(w)=0$ or for almost all words $w$ we have $\mathcal{L}_{{\mathcal{A}_{\infty}}}(w)=1$? The latter problem reduces to the expected separation problem as well as the distribution separation problem for $\mathcal{A}_{\infty}$.

6 Approximating LimAvg-automata in exponential time

In this section we develop algorithms for the approximate expected value and approximate distribution questions for (non-deterministic) LimAvg-automata. The presented algorithms work in exponential time in the size of the automaton, polynomial time in the size of the Markov chain and the precision.

The case of LimAvg-automata is significantly more complex than the other cases and hence we present the algorithms in stages. First, we restrict our attention to recurrent LimAvg-automata and the uniform distribution over infinite words. Recurrent automata are strongly connected with an appropriate set of initial states. We show that deterministic LimAvg-automata with bounded look-ahead approximate recurrent automata. Next, in Section 6.4 we extend this result to non-uniform measures given by Markov chains. Finally, in Section 7 we show the approximation algorithms for all (non-deterministic) LimAvg-automata and measures given by Markov chains.

Recurrent automata

Let $\mathcal{A}=(\Sigma,Q,Q_{0},\delta)$ be a non-deterministic LimAvg-automaton and $\widehat{\delta}$ be the extension of $\delta$ to all words $\Sigma^{*}$. The automaton $\mathcal{A}$ is recurrent if and only if the following conditions hold:

1. (1)

for every state $q\in Q$ there is a finite word $u$ such that $\widehat{\delta}(q,u)=Q_{0}$ ($\widehat{\delta}$ is the transition relation extended to words), and 2. (2)

for every set $S\subseteq Q$, if $\widehat{\delta}(Q_{0},w)=S$ for some word $w$, then there is a finite word $u$ such that $\widehat{\delta}(S,u)=Q_{0}$.

Intuitively, in recurrent automata $\mathcal{A}$, if two runs deviate at some point, with high probability it is possible to synchronize them. More precisely, for almost all words $w$, if $\pi$ is a run on $w$, and $\rho$ is a finite run up to position $i$, then $\rho$ can be extended to an infinite run that eventually coincides with $\pi$. Moreover, we show that with high probability, they synchronize within doubly-exponential number of steps in $|\mathcal{A}|$ (Lemma 6.3).

Example 6.1**.**

Consider the automaton depicted in Figure 1. This automaton is recurrent with the initial set of states $Q_{0}=\{q_{x},q_{a},q_{b}\}$. For condition (1) from the definition of recurrent automata, observe that for every state $q$ we have $\widehat{\delta}(q,abab)=Q_{0}$. For condition (2), observe that $\widehat{\delta}(Q_{0},b)=Q_{0}$, $\widehat{\delta}(Q_{0},a)=\{q_{a},q_{b}\}$ and $\widehat{\delta}(\{q_{a},q_{b}\},a)=\widehat{\delta}(\{q_{a},q_{b}\},b)=Q_{0}$. The automaton would also be recurrent in the case of $Q_{0}=\{q_{a},q_{b}\}$, but not in any other case.

Consider an automaton $\mathcal{A}$ depicted below:

$q_{L}$

$q_{R}$

a:0$$a:0

The automaton $\mathcal{A}$ is recurrent if the set of initial states is either $\{q_{L}\}$ or $\{q_{R}\}$, but not in the case of $\{q_{L},q_{R}\}$. Indeed, if we pick $q_{L}$ (resp., $q_{R}$) we can never reach the whole set $\{q_{L},q_{R}\}$. This realizes our intuition that runs that start in $q_{L}$ and $q_{R}$ will never synchronize.

We discuss properties of recurrent automata. For every $\mathcal{A}$ that is strongly connected as a graph there exists a set of initial states $T$ with which it becomes recurrent. Indeed, consider $\mathcal{A}$ as an unweighted $\omega$-automaton and construct a deterministic $\omega$-automaton $\mathcal{A}^{D}$ through the power-set construction applied to $\mathcal{A}$. Observe that $\mathcal{A}^{D}$ has a single bottom strongly-connected component (BSCC), i.e., a strongly connected component such that there are no transitions leaving that component. The set $Q_{0}$ belongs to that BSCC. Conversely, for any strongly connected automaton $\mathcal{A}$, if $Q_{0}$ belongs to the BSCC of $\mathcal{A}^{D}$, then $\mathcal{A}$ is recurrent.

Observe that for a recurrent automaton $\mathcal{A}$ the probability of words accepted by $\mathcal{A}$ is either [math] or $1$. Now, for a word $w$ consider a sequence of reachable sets of states $\Pi_{w}$ defined as $Q_{0},\widehat{\delta}(Q_{0},w[1]),\widehat{\delta}(Q_{0},w[2]),\ldots$ Since $\mathcal{A}^{D}$ has a single BSCC containing $Q_{0}$, all sets of $\Pi_{w}$ belong to that BSCC and hence either for almost all words $w$, the sequence $\Pi_{w}$ eventually contains only empty sets or for all words $w$, the sequence $\Pi_{w}$ consists of non-empty sets only. Observe that $\mathcal{A}$ has an infinite run on $w$ if and only if $\Pi_{w}$ consists of non-empty sets. It follows that the probability of the set of words having any infinite run in $\mathcal{A}$ is either [math] or $1$.

While Markov chains generate words letter by letter, to define a run of a word of the minimal value we need to have the completely generated word, i.e., the optimal transition at some position $i$ may depend on some positions $j>i$ in the word. This precludes application of standard techniques for probabilistic verification, which rely on the fact that the word and the run on it are generated simultaneously [33, 15, 4].

Key ideas

Our main idea is to change the non-determinism to bounded look-ahead. This must be inaccurate, as the expected value of a deterministic automaton with bounded look-ahead is always rational, whereas Theorem 3.3 shows that the values of non-deterministic automata may be irrational. Nevertheless, we show that bounded look-ahead is sufficient to approximate the probabilistic questions for recurrent automata (Lemma 6.5). Furthermore, the approximation can be done effectively (Lemma 6.12), which in turn gives us an exponential-time approximation algorithm for recurrent automata (Lemma 6.16). Then, we comment on the extension to all distributions given by Markov chains (Section 6.4). Finally, we show the proof for all LimAvg-automata over probability measures given by Markov chains (Theorem 7.1).

6.1 Nearly-deterministic approximations

Jumping runs

Let $k>0$ and let $N_{k}$ be the set of natural numbers not divisible by $k$. A $k$-jumping run $\xi$ of $\mathcal{A}$ on a word $w$ is an infinite sequence of states such that for every position $i\in N_{k}$ we have $(\pi[i-1],w[i],\pi[i])\in\delta$.

An $i$-th block of a $k$-jumping run is a sequence $\xi[ki,k(i+1)-1]$; within a block the sequence $\xi$ is consistent with transitions of $\mathcal{A}$. The positions $k,2k,\ldots\notin N_{k}$ are jump positions, where the sequence $\xi$ need not obey the transition relation of $\mathcal{A}$.

The cost ${C}$ of a transition of a $k$-jumping run $\xi$ within a block is defined as usual, while the cost of a jump is defined as the minimal weight of $\mathcal{A}$. The value of a $k$-jumping run $\xi$ is defined as the limit average computed for such costs.

Optimal and block-deterministic jumping runs

We say that a $k$-jumping run $\xi$ on a word $w$ is optimal if its value is the infimum over values of all $k$-jumping runs on $w$. We show that optimal $k$-jumping runs can be constructed nearly deterministically, i.e., only looking ahead to see the whole current block.

For every $S\subseteq Q$ and $u\in\Sigma^{k}$ we fix a run $\xi_{S,u}$ on $u$ starting in one of states of $S$, which has the minimal average weight. Then, given a word $w\in\Sigma^{\omega}$, we define a $k$-jumping run $\xi$ as follows. We divide $w$ into $k$-letter blocks $u_{1},u_{2},\ldots$ and we put $\xi=\xi_{S_{0},u_{1}}\xi_{S_{1},u_{2}}\ldots$, where $S_{0}=\{q_{0}\}$ and for $i>0$, $S_{i}$ is the set of states reachable from $q_{0}$ on the word $u_{1}\ldots u_{i}$. The run $\xi$ is a $k$-jumping run and it is indeed optimal. We call such runs block-deterministic — they can be constructed based on finite memory — the set of reachable states $S_{i}$ and the current block of the input word.

Since all runs of $\mathcal{A}$ are in particular $k$-jumping runs, the value of (any) optimal $k$-jumping run on $w$ is less or equal to $\mathcal{A}(w)$. We show that for recurrent LimAvg-automata, the values of $k$-jumping runs on $w$ converge to $\mathcal{A}(w)$ as $k$ tends to infinity. To achieve this, we construct a run of $\mathcal{A}$ which tries to “follow” a given jumping run, i.e., after almost all jump positions it is able to synchronize with the jumping run quickly.

Proof plan

Let $k>0$. Consider a word $w$ and some optimal $k$-jumping run $\xi_{o}$ on $w$. We construct a run $\pi_{f}$ of $\mathcal{A}$ in the following way. Initially, both runs start in some initial state $q_{0}$ and coincide. However, at the first jump position $\xi_{o}$ may take a move that is not a transition of $\mathcal{A}$. The run $\pi_{f}$ attempts to synchronize with $\xi_{o}$, i.e., to be at the same position in the same state, and then repeat transitions of $\xi_{o}$ until the end of the block. Then, in the next block, regardless of whether $\pi_{f}$ managed to synchronize with $\xi_{o}$ or not, we repeat the process. We say that a run $\pi_{f}$ constructed in such a way is a run following $\xi_{o}$.

In the following Lemma 6.3, we show that for $m\in\mathbb{N}$ large enough, with high probability, the run $\pi_{f}$ synchronizes with $\xi_{o}$ within $m$ steps. We then show that if $m$ is large enough and $k$ is much larger than $m$, then the values of runs $\pi_{f}$ and $\xi_{o}$ differ by less than $\epsilon$ (Lemma 6.5).

Let $q$ be a state of $\mathcal{A}$ and $u$ be a finite word. We say that a word $v$ saturates the pair $(q,u)$, if the set of reachable states from $q$ over $v$ equals all the states reachable over $uv$ from the initial states, i.e., $\widehat{\delta}(Q_{0},uv)=\widehat{\delta}(q,v)$.

Example 6.2**.**

Consider the automaton from Figure 1 with $Q_{0}=Q$. For any $(q,u)$, any word that contains the infix $abab$ saturates $(q,u)$, as $\widehat{\delta}(Q_{0},uv^{\prime}abab)=\widehat{\delta}(q,v^{\prime}abab)=Q_{0}$ for any $v^{\prime}$.

Observe that in the above, the probability that a random word of a length $4\ell$ does not saturate $(q,u)$ is bounded by $(1-\frac{1}{16})^{\ell}$. So the probability that a random word $v$ saturates $(q,u)$ quickly tends to $1$ with $|v|$. The next lemma shows that this is not a coincidence.

Lemma 6.3**.**

Let $\mathcal{A}$ be an NFA, $u$ be a finite word, and $q\in\widehat{\delta}(Q_{0},u)$. For every $\Delta>0$ there exists a natural number $\ell=2^{2^{O(|\mathcal{A}|)}}\log(\frac{1}{\Delta})$ such that over the uniform distribution on $\Sigma^{\ell}$ we have .

Proof 6.4**.**

First, observe that there exists a word $v$ saturating $(q,u)$. Let $S=\widehat{\delta}(Q_{0},u)$. Then, $q\in S$. Since $\mathcal{A}$ is recurrent, there exists a word $\alpha$ such that $Q_{0}=\widehat{\delta}(q,\alpha)$. It follows that $S=\widehat{\delta}(q,\alpha u)$. Since $q\in S$, we have $\widehat{\delta}(q,\alpha u)\subseteq\widehat{\delta}(S,\alpha u)$. It follows that for $i\geq 0$ we have $\widehat{\delta}(S,(\alpha u)^{i})=\widehat{\delta}(q,(\alpha u)^{i+1})\subseteq\widehat{\delta}(S,(\alpha u)^{i+1})$. Therefore, for some $i>0$ we have $\widehat{\delta}(q,(\alpha u)^{i})=\widehat{\delta}(S,(\alpha u)^{i})$, i.e., the word $(\alpha u)^{i}$ saturates $(q,u)$.

Now, we observe that there exists a saturating word that is exponentially bounded in $|\mathcal{A}|$. We start with the word $v_{0}$ equal $(\alpha u)^{i}$ and we pick any two positions $k<l$ such that $\widehat{\delta}(q,v_{0}[1,k])=\widehat{\delta}(q,v_{0}[1,l])$ and $\widehat{\delta}(S,v_{0}[1,k])=\widehat{\delta}(S,v_{0}[1,l])$. Observe that for $v_{1}$ obtained from $v_{0}$ by removal of $v[k+1,l]$, the reachable sets do not change, i.e., $\widehat{\delta}(q,v_{0})=\widehat{\delta}(q,v_{1})$ and $\widehat{\delta}(S,v_{0})=\widehat{\delta}(S,v_{1})$. We iterate this process until there are no such positions. The resulting word $v^{\prime}$ satisfies $\widehat{\delta}(S,v^{\prime})=\widehat{\delta}(q,v^{\prime})$. Finally, each position $k$ of $v^{\prime}$ defines the unique pair $(\widehat{\delta}(q,v_{0}[1,k]),\widehat{\delta}(S,v_{0}[1,k]))$ of subsets of $Q$. Therefore, the length of $v^{\prime}$ is bounded by $2^{2\cdot|Q|}$.

We have shown above that for every pair $(q,u)$ there exists a saturating word $v_{q,u}$ of length bounded by $N=2^{2\cdot|Q|}$. The probability of the word $v_{q,u}$ is $p_{0}=2^{-O(N)}$. Let $\ell=\frac{1}{p_{0}}\cdot\log(\frac{1}{\Delta})$; we show that the probability that $(q,u)$ is not saturated by a word from $\Sigma^{N\cdot\ell}$ is at most $\Delta$. Consider a word $x\in\Sigma^{N\cdot\ell}$. We can write it as $x=x_{1}\ldots x_{\ell}$, where all words $x_{k}$ have length $N$. If $x_{k}$ saturates $(q,ux_{1}\ldots x_{k-1})$, then $x_{1}\ldots x_{k}$ (as well as $x$) saturates $(q,u)$. Therefore, the word $x$ does not saturate $(q_{u})$ if for all $1\leq k\leq\ell$, $x_{k}$ does not saturate $(q,ux_{1}\ldots x_{k-1})$. The probability that $x\in\Sigma^{N\cdot\ell}$ does not saturate $(q,u)$ is at most $(1-p_{0})^{\ell}\leq(\frac{1}{2})^{\log(\frac{1}{\Delta})}\leq\Delta$.

Finally, we show that for almost all words the value of an optimal $k$-jumping run approximates the values of the word.

Lemma 6.5**.**

Let $\mathcal{A}$ be a recurrent LimAvg-automaton. For every $\epsilon\in\mathbb{Q}^{+}$, there exists $k$ such that for almost all words $w$, the value $\mathcal{A}(w)$ and the value of an optimal $k$-jumping run on $w$ differ by at most $\epsilon$. The value $k$ is doubly-exponential in $|\mathcal{A}|$ and polynomial in $\frac{1}{\epsilon}$.

Proof 6.6**.**

By Lemma 6.3, for all $\Delta>0$, $\ell=2^{2^{O(|\mathcal{A}|)}}\log(\frac{1}{\Delta})$, and all $k>\ell$, the probability that the run $\pi_{f}$ synchronizes with an optimal $k$-jumping run $\xi_{o}$ within $\ell$ steps in a block is at least $1-\Delta$.

Consider some $k>\ell$ and an optimal $k$-jumping run $\xi_{o}$ that is block-deterministic. Observe that the run $\pi_{f}$ of $\mathcal{A}$ following $\xi_{o}$ is also block-deterministic.

Consider a single block $\xi_{o}[i,i+k-1]$. By Lemma 6.3, the probability that $\pi_{f}[i+\ell-1]=\xi_{o}[i+\ell-1]$ is at least $1-\Delta$. In such a case, the sum of costs on that block of $\pi_{f}$ exceeds $\xi_{o}$ by at most $D\cdot\ell$, where $D$ is the difference between the maximal and the minimal weight in $\mathcal{A}$. Otherwise, if $\pi_{f}$ does not synchronize, we bound the difference of the sums of values on that block by the maximal possible difference $D\cdot k$.

Since runs are block-deterministic, synchronization of $\pi_{f}$ and $\xi_{o}$ satisfies the Markov property; it depends only on the current block and the set of states $S$ reachable on the input word until the beginning of the current block. We observe that as $\mathcal{A}$ is recurrent, the corresponding Markov chain, whose states are reachable sets of states $S$ of $\mathcal{A}$, has only a single BSCC. Therefore, for almost all words, the average ratio of $k$-element blocks, in which $\pi_{f}$ synchronizes with $\xi_{o}$ within $\ell$ steps, is $1-\Delta$. We then conclude that for almost all words the difference between $\pi_{f}$ and $\xi_{o}$ is bounded by $\gamma=\frac{(1-\Delta)\cdot(D\cdot\ell)+\Delta\cdot(D\cdot k)}{k}$. Observe that with $\Delta<\frac{\epsilon}{2\cdot D}$ and $k>\frac{2\cdot D\cdot\ell}{\epsilon}$, the value $\gamma$ is less than $\epsilon$.

6.2 Random variables

Given a recurrent LimAvg-automaton $\mathcal{A}$ and $k>0$, we define a function $g[{k}]:\Sigma^{\omega}\to\mathbb{R}$ such that $g[{k}](w)$ is the value of some optimal $k$-jumping run $\xi_{o}$ on $w$. We can pick $\xi_{o}$ to be block-deterministic and hence $g[{k}]$ corresponds to a Markov chain $M[k]$. More precisely, we define $M[k]$ labeled by $\Sigma^{k}$ such that for every word $w$, the limit average of the path in $M[k]$ labeled by blocks of $w$ (i.e., blocks $w[1,k]w[k+1,2k]\ldots$) equals $g[{k}](w)$. Moreover, the distribution of blocks $\Sigma^{k}$ is uniform and hence $M[k]$ corresponds to $g[{k}]$ over the uniform distribution over $\Sigma$. The Markov chain $M[k]$ is a labeled weighted Markov chain [18], such that its states are all subsets of $Q$, the set of states of $\mathcal{A}$. For each state $S\subseteq Q$ and $u\in\Sigma^{k}$, the Markov chain $\mathcal{M}$ has an edge $(S,\widehat{\delta}(S,u))$ of probability $\frac{1}{|\Sigma|^{k}}$. The weight of an edge $(S,S^{\prime})$ labeled by $u$ is the minimal average of weights of any run from some state of $S$ to some state of $S^{\prime}$ over the word $w$.

We have the following:

Lemma 6.7**.**

Let $\mathcal{A}$ be a recurrent LimAvg-automaton and $k>0$. (1) The functions $g[{k}]$ and $\mathcal{L}_{{\mathcal{A}}}$ are random variables. (2) For almost all words $w$ we have $g[{k}](w)=\mathbb{E}(g[{k}])$ and $\mathcal{L}_{{\mathcal{A}}}(w)=\mathbb{E}(\mathcal{L}_{{\mathcal{A}}})$.

Proof 6.8**.**

Since $\mathcal{A}$ is recurrent, $M[k]$ has a single BSCC and hence $M[k]$ and $g[{k}]$ return the same value for almost all words [18]. This implies that the preimage through $g[{k}]$ of each set has measure [math] or $1$, and hence $g[{k}]$ is measurable [17]. Lemma 6.5 implies that (measurable functions) $g[{k}]$ converge to $\mathcal{L}_{{\mathcal{A}}}$ with probability $1$, and hence $\mathcal{L}_{{\mathcal{A}}}$ is measurable [17]. As the limit of $g[{k}]$, $\mathcal{L}_{{\mathcal{A}}}$ also has the same value for almost all words.

Remark 6.9**.**

The automaton $\mathcal{A}$ from the proof of Theorem 3.3 is recurrent (it resets after each \$$), so the value of \mathcal{A}$on almost all words is irrational. Yet, for every ultimately periodic word$vw^{\omega}$, the value of$\mathcal{A}$ is rational. This means that while the expected value is realized by almost all words, it is not realized by any ultimately periodic word.

6.3 Approximation algorithms

We show that the expected value of $g[{k}]$ can be efficiently approximated. The approximation is exponential in the size of $\mathcal{A}$, but only logarithmic in $k$ (which is doubly-exponential due to Lemma 6.5).

To approximate the expected value of $g[{k}]$ we need to compute the expected value of $\mathcal{A}$ over $k$-letter blocks. Such blocks are finite and hence we consider $\mathcal{A}$ as a finite-word automaton with the average value function Avg. More precisely, for $S$ being a subset of states of $\mathcal{A}$, we define $\mathcal{A}_{S}^{\mathrm{fin}}$ as a Avg-automaton over finite words as $\mathcal{A}$, which initial states set to $S$ and all states accepting. We can approximate the expected value of $\mathcal{A}_{S}^{\mathrm{fin}}$ over words $\Sigma^{k}$ in logarithmic time in $k$.

Lemma 6.10**.**

Let $\mathcal{A}$ be a recurrent LimAvg-automaton, let $S,S^{\prime}$ be subsets of states of $\mathcal{A}$, and let $i>0$. We can approximate the expected value $\mathbb{E}(\{\mathcal{A}_{S}^{\mathrm{fin}}(w)\mid|w|=2^{k}$ and $\widehat{\delta}(S,w)=S^{\prime}\})$ within a given $\epsilon\in\mathbb{Q}^{+}$ in exponential time in $|\mathcal{A}|$ and polynomial time in $k$ and $\frac{1}{\epsilon}$.

Proof 6.11**.**

Let $h(q,w,q^{\prime})$ be the infimum average weight over runs from $q$ to $q^{\prime}$ over $w$. Consider $\epsilon_{0}=\frac{\epsilon}{k+1}$. Let $H=\{j\cdot\epsilon_{0}\mid j\in\mathbb{Z}\}\cap(-|\mathcal{A}|,|\mathcal{A}|)$ be a finite set and let $\left[x\right]_{\epsilon_{0}}$ stand for the greatest number from $H$ not exceeding $x$.

Consider $i\in\{0,\dots,k\}$ and let $N=2^{i}$. We define a function $\tilde{h}^{i}:Q\times\Sigma^{N}\times Q\to H$ as follows. First, we define $\tilde{h}^{0}(q,w,q^{\prime})=\left[h(q,w,q^{\prime})\right]_{\epsilon_{0}}$. Then, inductively, we define

[TABLE]

We show by induction on $i$ that for all $i$, $q$, $q^{\prime}$, $N=2^{i}$ and $w\in\Sigma^{N}$ we have $|h(q,w,q^{\prime})-\tilde{h}^{i}(q,w,q^{\prime})|\leq(i+1)\epsilon_{0}$. First, we comment on the deteriorating precision. Notice that $|h(q,w,q^{\prime})-\tilde{h}^{i}(q,w,q^{\prime})|\leq\epsilon_{0}$ may not hold in general. Let us illustrate this with a simple toy example. Consider $\epsilon_{0}=1$, $x\in(0,1)$ and $y\in(1,2)$. Then $\frac{x+y}{2}\in(\frac{1}{2},\frac{3}{2})$, thus $\left[\frac{x+y}{2}\right]_{\epsilon_{0}}\in\{0,1\}$. However, knowing only $\left[x\right]_{\epsilon_{0}}$ and $\left[y\right]_{\epsilon_{0}}$, we cannot asses whether the answer should be [math] or $1$. Therefore, when iterating the above-described procedure, we may lose some precision (up to one $\epsilon_{0}$ at each step); this is why we start with $\epsilon_{0}$ rather than $\epsilon$.

Now, we show by induction $|h(q,w,q^{\prime})-\tilde{h}^{i}(q,w,q^{\prime})|\leq(i+1)\epsilon_{0}$. More precisely, we show that

1. (1)

$\tilde{h}^{i}(q,w,q^{\prime})\leq h(q,w,q^{\prime})$* and* 2. (2)

$h(q,w,q^{\prime})-\tilde{h}^{i}(q,w,q^{\prime})\leq(i+1)\epsilon_{0}$.

The case $i=0$ follows from the definition of $\tilde{h}^{0}(q,w,q^{\prime})$. Consider $i>0$ and assume that for all words $w$ of length $2^{i}$ the induction hypothesis holds. Consider $w=w_{1}w_{2}$ and states $q,q^{\prime}$. There exists $q^{\prime\prime}$ such that $h(q,w,q^{\prime})=h(q,w_{1},q^{\prime\prime})+h(q^{\prime\prime},w_{1},q^{\prime})$. Then, due to induction assumption of (1) we have $\tilde{h}^{i-1}(q,w_{1},q^{\prime\prime})\leq h(q,w_{1},q^{\prime\prime})$ and $\tilde{h}^{i-1}(q^{\prime\prime},w_{2},q^{\prime})\leq h(q^{\prime\prime},w_{2},q^{\prime})$. In consequence, we get (1).

Now, to show (2), consider a state $s$ that realizes the minimum from the definition of $\tilde{h}^{i}(q,w,q^{\prime})$. There are numbers $a,b\in\mathbb{Z}$ such that $\tilde{h}^{i-1}(q,w_{1},s)=a\epsilon_{0}$, and $\tilde{h}^{i-1}(s,w_{2},q^{\prime})=b\epsilon_{0}$.

Then, $h(q,w,q^{\prime})\leq\frac{h(q,w_{1},s)+h(s,w_{2},q)}{2}$ and we have

[TABLE]

Observe that $\left[\frac{(a+b)\epsilon_{0}}{2}\right]_{\epsilon_{0}}=\frac{(a+b)\epsilon_{0}}{2}$ if $a+b$ is even and $\left[\frac{(a+b)\epsilon_{0}}{2}\right]_{\epsilon_{0}}=\frac{(a+b)\epsilon_{0}}{2}-\frac{\epsilon_{0}}{2}$ otherwise. This gives us the following inequality

[TABLE]

Due to the induction hypothesis (2) we have $h(q,w_{1},q^{\prime\prime})-a\epsilon_{0}\leq i\epsilon_{0}$ $h(q^{\prime\prime},w_{2},q^{\prime})-b\epsilon_{0}\leq i\epsilon_{0}$ and it gives us (2).

We cannot compute the functions $\tilde{h}^{i}$ directly (in reasonable time), because there are too many words to be considered. However, we can compute them symbolically. Define the clusterization function $c^{i}$ as follows. Let $N=2^{i}$. For each function $f\colon Q\times\Sigma^{N}\times Q\to H$ we define $c^{i}(f)=|\{w\mid\forall q,q^{\prime}.\tilde{h}^{i}(q,w,q^{\prime})=f(q,w,q^{\prime})\}|$. Basically, for each function $f$, clasterization counts the number of words realizing $f$ though functions $\tilde{h}^{i}(\cdot,w,\cdot)$.

The function $c^{0}$ can be computed directly. Then, $c^{i+1}(f)$ can be computed as the sum of $c^{i}(f_{1})\cdot c^{i}(f_{2})$ over all the functions $f_{1},f_{2}$ such that $f=f_{1}*f_{2}$, where $h_{1}*h_{2}(q,q^{\prime\prime})=\left[\min_{q^{\prime}\in F_{2}}\frac{h_{1}(q,q^{\prime})+h_{2}(q^{\prime},q^{\prime\prime})}{2}\right]_{\epsilon_{0}}$.

It follows that we can compute the $k$-clusterization in time exponential in $|\mathcal{A}|$, polynomial in $\frac{1}{\epsilon}$ and $k$. The desired expected valued can be derived from the $k$-clusterization in the straightforward way.

In consequence, we can approximate the expected value of $g[{k}]$ in exponential time in $|\mathcal{A}|$ but logarithmic in $k$, which is important as $k$ may be doubly-exponential in $|\mathcal{A}|$ (Lemma 6.5).

Lemma 6.12**.**

Given a recurrent LimAvg-automaton $\mathcal{A}$, $k=2^{l}$ and $\epsilon\in\mathbb{Q}^{+}$, the expected value $\mathbb{E}(g[{k}])$ can be approximated up to $\epsilon$ in exponential time in $|\mathcal{A}|$, logarithmic time in $k$ and polynomial time in $\frac{1}{\epsilon}$.

Proof 6.13**.**

Recall that the expected values of $M[k]$ and $g[{k}]$ coincide. Observe that $M[k]$ can be turned into a weighted Markov chain $N[k]$ over the same set of states with one edge between any pair of states as follows. For an edge $(S,S^{\prime})$, we define its probability as $\frac{1}{|\Sigma|^{k}}$ multiplied by the number of edges from $S$ to $S^{\prime}$ with positive probability in $M[k]$ and the weight of $(S,S^{\prime})$ in $N[k]$ is the average of the weights of all such the edges in $M[k]$, i.e., the weight of $(S,S^{\prime})$ is $\mathbb{E}(\{\mathcal{A}_{S}^{\mathrm{fin}}(w)\mid w\in\Sigma^{k}$ and $\widehat{\delta}(S,w)=S^{\prime}\})$ (see Lemma 6.10). Observe that the expected values of $M[k]$ and $N[k]$ coincide.

Having the Markov chain $N[k]$, we can compute its expected value in polynomial time [18]. Since $N[k]$ has the exponential size in $|\mathcal{A}|$, we can compute it in exponential time in $|\mathcal{A}|$. However, we need to show how to construct $N[k]$. In particular, computing $\mathbb{E}(\{\mathcal{A}_{S}^{\mathrm{fin}}(w)\mid w\in\Sigma^{k}$ and $\widehat{\delta}(S,w)=S^{\prime}\})$ can be computationally expensive as $k$ can be doubly-exponential in $|\mathcal{A}|$ (Lemma 6.5). Still, due to Lemma 6.10, we can approximate $\mathbb{E}(\{\mathcal{A}_{S}^{\mathrm{fin}}(w)\mid w\in\Sigma^{k}$ and $\widehat{\delta}(S,w)=S^{\prime}\})$ in exponential time in $|\mathcal{A}|$, logarithmic time in $k$ and polynomial time in $\epsilon$. Therefore, we can compute a Markov chain $\mathcal{M}^{\approx}$ with the same structure as $N[k]$ and such that for every edge $(S,S^{\prime})$ the weight of $(S,S^{\prime})$ in $\mathcal{M}^{\approx}$ differs from the weight in $N[k]$ by at most $\epsilon$. Therefore, the expected values of $\mathcal{M}^{\approx}$ and $N[k]$ differ by at most ${\epsilon}$.

Lemma 6.5 and Lemma 6.12 give us approximation algorithms for the expected value and the distribution of recurrent automata over the uniform distribution:

Lemma 6.14**.**

Given a recurrent LimAvg-automaton $\mathcal{A}$, $\epsilon\in\mathbb{Q}^{+}$ and $\lambda\in\mathbb{Q}$, we can compute $\epsilon$-approximations of the distribution $\mathbb{D}_{\mathcal{A}}(\lambda)$ and the expected value $\mathbb{E}(\mathcal{A})$ with respect to the uniform measure in exponential time in $|\mathcal{A}|$ and polynomial time in $\frac{1}{\epsilon}$.

Proof 6.15**.**

For uniform distributions, by Lemma 6.5, for every $\epsilon>0$, there exists $k$ such that $|\mathbb{E}(\mathcal{A})-\mathbb{E}(g[{k}])|\leq\frac{\epsilon}{2}$. The value $k$ is doubly-exponential in $|\mathcal{A}|$ and polynomial in $\frac{1}{\epsilon}$. Then, by Lemma 6.12, we can compute $\gamma$ such that $|\gamma-\mathbb{E}(g[{k}])|\leq\frac{\epsilon}{2}$ in exponential time in $|\mathcal{A}|$ and polynomial in $\frac{1}{\epsilon}$. Thus, $\gamma$ differs from $\mathbb{E}(\mathcal{A})$ by at most $\epsilon$. Since almost all words have the same value, we can approximate $\mathbb{D}_{\mathcal{A}}(\lambda)$ by comparing $\lambda$ with $\gamma$, i.e., $1$ is an $\epsilon$-approximation of $\mathbb{D}_{\mathcal{A}}(\lambda)$ if $\lambda\leq\gamma$, and otherwise [math] is an $\epsilon$-approximation of $\mathbb{D}_{\mathcal{A}}(\lambda)$.

6.4 Non-uniform measures

We briefly discuss how to adapt Lemma 6.14 to all measures given by Markov chains. We sketch the main ideas.

Key ideas. Assuming that (a variant of) Lemma 6.3 holds for any probability measure given by a Markov chain, the proofs of Lemmas 6.5, 6.12 and 6.16 can be easily adapted. Therefore we focus on adjusting Lemma 6.3.

Observe that if a Markov chain $\mathcal{M}$ produces all finite prefixes $u\in\Sigma^{*}$ with non-zero probability, then the proof of Lemma 6.3 can be straightforwardly adapted. Otherwise, if some finite words cannot be produced by a Markov chain $\mathcal{M}$, then Lemma 6.3 may be false. However, if there are words $w$ such that $\mathcal{A}$ has an infinite run on $w$, but $\mathcal{M}$ does not emit $w$, we can restrict $\mathcal{A}$ to reject such words. Therefore, we assume that for every word $w$, if $\mathcal{A}$ has an infinite run on $w$, then $\mathcal{M}$ has an infinite path with non-zero probability on $w$ ($\mathcal{M}$ emits $w$). Then, the current proof of Lemma 6.3 can be straightforwardly adapted to the probability measure given by $\mathcal{M}$. In consequence, we can compute $\mathbb{D}_{\mathcal{M},\mathcal{A}}(\lambda)$ and $\mathbb{E}_{\mathcal{M}}(\mathcal{A})$ in exponential time in $|\mathcal{A}|$ and polynomial time in $|\mathcal{M}|$ and $\frac{1}{\epsilon}$.

More precisely, we first observe that we may enforce $\mathcal{M}$ to be “deterministic”, i.e., for all states $s$ and letters $a$ at most one outgoing transition labeled with $a$ has positive probability. We can determinise $\mathcal{M}$ by extending the alphabet $\Sigma$ to $\Sigma\times S$, where $S$ is the set of states of $\mathcal{M}$. The second component in $\Sigma\times S$ encodes the target state in the transition. Observe that $\mathcal{A}$ can be extended to the corresponding automaton $\mathcal{A}^{\prime}$ over $\Sigma\times S$ by cloning transitions, i.e., for every transition $(q,a,q^{\prime})$, the automaton $\mathcal{A}^{\prime}$ has transitions $(q,(a,s),q^{\prime})$ for every $s\in S$ (i.e., $\mathcal{A}^{\prime}$ ignores the state of $\mathcal{M}$). For such a deterministic Markov chain $\mathcal{M}^{\prime}$, we define a deterministic $\omega$-automaton $\mathcal{A}_{\mathcal{M}}$ that accepts words emitted by $\mathcal{M}^{\prime}$. Finally, we consider the automaton $\mathcal{A}^{R}=\mathcal{A}_{\mathcal{M}}\times\mathcal{A}^{\prime}$, which has infinite runs only on words that are emitted by $\mathcal{M}^{\prime}$. Therefore, as we discussed, we can adapt the proof of Lemma 6.14 in such a case and compute $\mathbb{D}_{\mathcal{M}^{\prime},\mathcal{A}^{R}}(\lambda)$ and $\mathbb{E}_{\mathcal{M}^{\prime}}(\mathcal{A}^{R})$ (in exponential time in $|\mathcal{A}^{R}|$, polynomial time in $|\mathcal{M}|$ and $\frac{1}{\epsilon}$; notice that $|\mathcal{A}^{R}|$ is polynomial in $|\mathcal{A}|$). Finally, observe that $\mathbb{D}_{\mathcal{M},\mathcal{A}}(\lambda)=\mathbb{D}_{\mathcal{M}^{\prime},\mathcal{A}^{R}}(\lambda)$ and $\mathbb{E}_{\mathcal{M}}(\mathcal{A})=\mathbb{E}_{\mathcal{M}^{\prime}}(\mathcal{A}^{R})$. In consequence, we have the following:

Lemma 6.16**.**

Given a recurrent LimAvg-automaton $\mathcal{A}$, Markov chain $\mathcal{M}$, $\epsilon\in\mathbb{Q}^{+}$ and $\lambda\in\mathbb{Q}$, we can compute $\epsilon$-approximations of the distribution $\mathbb{D}_{\mathcal{M},\mathcal{A}}(\lambda)$ and the expected value $\mathbb{E}_{\mathcal{M}}(\mathcal{A})$ in exponential time in $|\mathcal{A}|$ and polynomial time in $|\mathcal{M}|$ and $\frac{1}{\epsilon}$.

7 Non-recurrent automata

We present the approximation algorithms for all non-deterministic LimAvg-automata over measures given by Markov chains.

Theorem 7.1**.**

(1) For a non-deterministic LimAvg-automaton $\mathcal{A}$ the function $\mathcal{L}_{{\mathcal{A}}}:\Sigma^{\omega}\to\mathbb{R}$ is measurable. (2) Given a non-deterministic LimAvg-automaton $\mathcal{A}$, Markov chain $\mathcal{M}$, $\epsilon\in\mathbb{Q}^{+}$, and $\lambda\in\mathbb{Q}$, we can $\epsilon$-approximate the distribution $\mathbb{D}_{\mathcal{M},\mathcal{A}}(\lambda)$ and the expected value $\mathbb{E}(\mathcal{A})$ in exponential time in $|\mathcal{A}|$ and polynomial time in $|\mathcal{M}|$ and $\frac{1}{\epsilon}$.

Proof 7.2**.**

Consider $\mathcal{A}$ as an $\omega$-automaton. It has no acceptance conditions and hence we can determinise it with the standard power-set construction to a deterministic automaton $\mathcal{A}^{D}$. Then, we construct a Markov chain $\mathcal{M}\times\mathcal{A}^{D}$, compute all its BSCCs $R_{1},\ldots,R_{k}$ along with the probabilities $p_{1},\ldots,p_{k}$ of reaching each of these sets. This can be done in polynomial time in $\mathcal{M}\times\mathcal{A}^{D}$ [18, 4], and hence polynomial in $\mathcal{M}$ and exponential in $\mathcal{A}$. Let $H_{1},\ldots,H_{k}$ be sets of paths in $\mathcal{M}\times\mathcal{A}^{D}$ such that for each $i$, all $\rho\in H_{i}$ eventually reach $R_{i}$ and stay there forever. Observe that each $H_{i}$ is a Borel set; the set $H_{i}^{p}$ of paths that stay in $R_{i}$ past position $p$ is closed and $H_{i}=\bigcup_{p\geq 0}H_{i}^{p}$. It follows that each $H_{i}$ is measurable. We show how to compute an $\epsilon$-approximation of the conditional expected value $\mathbb{E}_{\mathcal{M}}(\mathcal{A}\mid H_{i})$.

Consider a BSCC $R_{i}$. The projection of $R_{i}$ on the first component $R_{i}^{1}$ is a BSCC in $\mathcal{M}$ and the projection on the second component $R_{i}^{2}$ is an SCC of $\mathcal{A}_{D}$. Let $(s,A)\in R_{i}$. If we fix $s$ as the initial state of $R_{i}^{1}$ and $A$ as the initial state of $R_{i}^{2}$, then $R_{i}$ are all reachable states of $R_{i}^{1}\times R_{i}^{2}$. The set $R_{i}^{2}$ consists of the states of $\mathcal{A}^{D}$, which are subsets of states of $\mathcal{A}$. Therefore, the union $\bigcup R_{i}^{2}$ is a subset of states of $\mathcal{A}$ and it consists of some SCCs $S_{1},\ldots,S_{m}$ of $\mathcal{A}$. All these SCCs are reachable, but it does not imply that there is a run of $\mathcal{A}$ that stays in $S_{j}$ forever. We illustrate that in the following example.

Consider the automaton $\mathcal{A}$ presented in Figure 3, where $q_{I}$ is the initial state, and a single-state Markov chain $\mathcal{M}$ generating uniform distribution.

Then, all paths in $\mathcal{M}\times\mathcal{A}^{D}$ are eventually contained in $\mathcal{M}\times\{Q\}$, i.e., the second component consists of all states of $\mathcal{A}$. Still, if a word $w$ has infinitely many letters $a$, then $\mathcal{A}$ has no (infinite) run on $w$ that visits the state $q_{F}$. The set of infinite words that contain finitely many letters $a$ is countable and hence has probability [math]. Therefore, almost all words (i.e., all except for some set of probability [math]) have no run that visits the state $q_{F}$.

To avoid such pathologies, we divide SCCs into two types: permanent and transitory. More precisely, for a path $\rho$ in $\mathcal{M}\times\mathcal{A}^{D}$ let $w_{\rho}$ be the word labeling $\rho$. We show that for each SCC $S_{j}$, one of the following holds:

• •

$S_{j}$* is permanent, i.e., for almost all paths $\rho\in H_{i}$ (i.e., the set of paths of probability $1$), the automaton $\mathcal{A}$ has a run on the word $w_{\rho}$ that eventually stays in $S_{j}$ forever, or*

• •

$S_{j}$* is transitory, i.e., for almost all paths $\rho\in H_{i}$, the automaton $\mathcal{A}$ has no run on $w_{\rho}$ that eventually stays in $S_{j}$.*

Consider an SCC $S_{j}$. If $S_{j}$ is permanent, then it is not transitory. We show that if $S_{j}$ is not permanent, then it is transitory. Suppose that $S_{j}$ is not permanent and consider any $(s,A)\in H_{i}$. Almost all paths in $H_{i}$ visit $(s,A)$ and since $S_{j}$ in not permanent, there exists an infinite path $\rho$ that visits $(s,A)$ and $\mathcal{A}$ has no run on $w_{\rho}$ that stays in $S_{j}$ forever. Let $u$ be the suffix of $w_{\rho}$ that labels $\rho$ past some occurrence of $(s,A)$. We observe that $\widehat{\delta}(A\cap S_{j},u)=\emptyset$ and hence for some finite prefix $u^{\prime}$ of $u$ we have $\widehat{\delta}(A\cap S_{j},u^{\prime})=\emptyset$. Let $p$ be the probability that $\mathcal{M}\times\mathcal{A}^{D}$ in the state $(s,A)$ generates a path labeled by $u^{\prime}$. The probability that a path that visits $(s,A)$ at least $\ell$ times does not contain $(s,A)$ followed by labels $u^{\prime}$ is at most $(1-p)^{\ell}$. Observe that for almost all paths in $H_{i}$, the state $(s,A)$ is visited infinitely often and hence almost all paths contain $(s,A)$ followed by labels $u^{\prime}$ upon which the path leaves $S_{j}$. Therefore, $S_{j}$ is transitory.

To check whether $S_{j}$ is permanent or transitory, observe that for any $(s,A)\in H_{i}$, in the Markov chain $\mathcal{M}\times\mathcal{A}^{D}$, we can reach the set $\mathcal{M}\times\{\emptyset\}$ from $(s,A\cap S_{j})$ if and only if $S_{j}$ is transitory. The former condition can be checked in polynomial space.

We mark each SCC $S_{1},\ldots,S_{k}$ as permanent or transitory and for every permanent SCC $S_{j}$, we compute an $\epsilon$-approximation of $\mathbb{E}_{\mathcal{M}}(\mathcal{A}[S_{j}]\mid H_{i})$, which is the expected value of $\mathcal{A}$ under condition $H_{i}$ with the restriction to runs that eventually stay in $S_{j}$. Observe that an $\epsilon$-approximation of $\mathbb{E}_{\mathcal{M}}(\mathcal{A}[S_{j}]\mid H_{i})$ can be computed using Lemma 6.16. Indeed, we pick $(s,A)\in H_{i}$ and observe that $\mathcal{A}$ restricted to states $S_{j}$ is recurrent (with an appropriate initial states). Finally, we pick the minimum $\gamma$ over the computed expected values $\mathbb{E}_{\mathcal{M}}(\mathcal{A}[S_{j}]\mid H_{i})$ and observe that almost all words in $H_{i}$ have value $\gamma$. It follows that $\mathbb{E}_{\mathcal{M}}(\mathcal{A}\mid H_{i})=\gamma$.

In each BSCC $R_{i}$, almost all words have value $\mathbb{E}_{\mathcal{M}}(\mathcal{A}\mid H_{i})$. As we discussed earlier, each $H_{i}$ is measurable, and hence the function $\mathcal{L}_{{\mathcal{A}}}:\Sigma^{\omega}\to\mathbb{R}$ is measurable. Moreover, to approximate the distribution $\mathbb{D}_{\mathcal{M},\mathcal{A}}(\lambda)$, we sum probabilities of $p_{i}$ of reaching the BSCCs $R_{i}$ over $R_{i}$’s such that the $\epsilon$-approximation of $\mathbb{E}_{\mathcal{M}}(\mathcal{A}\mid H_{i})$ is less or equal to $\lambda$. Finally, we compute an $\epsilon$-approximation of $\mathbb{E}_{\mathcal{M}}(\mathcal{A})$ from $\epsilon$-approximations of conditional expected values $\mathbb{E}_{\mathcal{M}}(\mathcal{A}\mid H_{i})$ using the identity $\mathbb{E}_{\mathcal{M}}(\mathcal{A})=\sum_{i=1}^{k}p_{i}\cdot\mathbb{E}_{\mathcal{M}}(\mathcal{A}\mid H_{i})$.

8 Determinising and approximating LimAvg-automata

For technical simplicity, we assume that the distribution of words is uniform. However, the results presented here extend to all distributions given by Markov chains.

Recall that for the LimAvg automata, the value of almost all words (i.e., all except for some set of words of probability [math]) whose optimal runs end up in the same SSC, is the same. This means that there is a finite set of values (not greater than the number of SSCs of the automaton) such that almost all words have their values in this set.

LimAvg-automata are not determinisable [11]. We say that a non-deterministic LimAvg-automaton $\mathcal{A}$ is weakly determinisable if there is a deterministic LimAvg-automaton $\mathcal{B}$ such that $\mathcal{A}$ and $\mathcal{B}$ have the same value over almost all words. From [12] we know that deterministic automata return rational values for almost all words, so not all LimAvg-automata are weakly determinisable. However, we can show the following.

Theorem 8.1**.**

A LimAvg-automaton $\mathcal{A}$ is weakly determinisable if and only if it returns rational values for almost all words.

Proof 8.2** (Proof sketch).**

Assume an automaton $\mathcal{A}$ with SSCs $C_{1},\dots,C_{m}$. For each $i$ let $v_{i}$ be defined as the expected value of $\mathcal{A}$ when its set of initial states is $C_{i}$ and the run is bounded to stay in $C_{i}$. If $\mathcal{A}$ has no such runs for some $C_{i}$, then $v_{i}=\infty$.

We now construct a deterministic automaton $B$ with rational weights using the standard power-set construction. We define the cost function such that the cost of any transition from a state $Y$ is the minimal value $v_{i}$ such that $v_{i}$ is rational and $Y$ contains a state from $C_{i}$. If there are no such $v_{i}$, then we set the cost to the maximal cost of $\mathcal{A}$. Roughly speaking, $B$ tracks in which SSCs $A$ can be and the weight corresponds to the SSC with the lowest value.

To see that $B$ weakly determinises $A$ observe that for almost all words $w$, a run with the lowest value over $w$ ends in some SSC and its value then equals the expected value of this component, which is rational as the value of this word is rational.

A straightforward corollary is that every non-deterministic LimAvg-automaton can be weakly determinised by an LimAvg-automaton with real weights.

Theorem 8.1 does not provide an implementable algorithm for weak-determinisation, because of the hardness of computing the values $v_{i}$. It is possible, however, to approximate this automaton. We say that a deterministic LimAvg-automaton $B$ $\epsilon$-approximates $\mathcal{A}$ if for almost every word $w$ we have that $\mathcal{L}_{{B}}(w)\in[\mathcal{L}_{{\mathcal{A}}}(w)-\epsilon,\mathcal{L}_{{\mathcal{A}}}(w)+\epsilon]$.

Theorem 8.3**.**

For every $\epsilon>0$ and a non-deterministic LimAvg-automaton $\mathcal{A}$, one can compute in exponential time a deterministic LimAvg-automaton that $\epsilon$-approximates $\mathcal{A}$.

The proof of this theorem is similar to the proof of Theorem 8.1, except now it is enough to approximate the values $v_{i}$, which can be done in exponential time.

Acknowledgements

Our special thanks go to Günter Rote who pointed out an error in an earlier version of our running example.

\doclicenseThis