Finitely dependent processes are finitary

TL;DR

The paper proves that finitely dependent invariant processes on transitive amenable graphs can be represented as finitary factors of i.i.d. processes, with entropy arbitrarily close under certain geometric conditions.

Contribution

It establishes a new representation theorem linking finitely dependent processes to i.i.d. processes on specific graphs, answering a question of Holroyd.

Findings

Finitely dependent invariant processes are finitary factors of i.i.d. processes.

Under certain geometric conditions, the entropy of the i.i.d. process can be made arbitrarily close.

Provides an affirmative answer to Holroyd's question.

Abstract

We show that any finitely dependent invariant process on a transitive amenable graph is a finitary factor of an i.i.d. process. With an additional assumption on the geometry of the graph, namely that no two balls with different centers are identical, we further show that the i.i.d. process may be taken to have entropy arbitrarily close to that of the finitely dependent process. As an application, we give an affirmative answer to a question of Holroyd.

Equations

$${\Lambda }_r(u)\ne {\Lambda }_r(v)\text{for any distinct }u,v\in \mathbb{V}\text{ and }r\ge 0,$$

$$H(Z):=-\sum _z\mathbb{P}(Z=z)\log \mathbb{P}(Z=z),$$

$$h(X):=\underset{\begin{array}{c}V\subset \mathbb{V}\text{ finite}\\ \text{and non-empty}\end{array}}{\inf }\frac{H(X_V)}{|V|}.$$

$$\underset{n\to \infty }{\lim }\frac{|\partial F_n|}{|F_n|}=0.$$

$$h(X)=\underset{n\to \infty }{\lim }\frac{H(X_{F_n})}{|F_n|}\text{for any Følner sequence }(F_n{)}_{n=1}^{\infty }\text{ in }G.$$

$$\frac{H(X_F)}{|F|}\le h(X)+ϵ\text{whenever }F\subset \mathbb{V}\text{ is non-empty and finite and }|\partial F|\le \delta |F|.$$

$$\frac{H(X_F\mid \mathcal{E})}{|F|}\le \log |S|\begin{array}{c}\text{whenever }F\subset \mathbb{V}\text{ is non-empty and finite}\\ \text{and E is an event with positive probability}\end{array},$$

$${\Gamma }_{u,v}:=\{\gamma \in \Gamma :\gamma u=v\}.$$

$$|{\Gamma }_{u,u}v|=|{\Gamma }_{v,v}u|\text{for all }u,v\in \mathbb{V}.$$

$$\sum _{u}f(u,0)=\sum _{v}f(0,v)\text{for any diagonally Γ-invariant function }f:{\mathbb{V}}^2\to [0,\infty ].$$

$$\frac{|\partial E|}{|E|}\le \frac{|\partial F|}{|F|}\text{and}E={\Gamma }_{0}E.$$

$$E:=\bigcup _{i:U\le p_i}{V}_i,\text{where }{p}_i:=\frac{|F\cap V_i|}{|V_i|}.$$

$$\mathbb{E}|E|=\sum _i\mathbb{P}(U\le p_i)\cdot |V_i|=\sum _i{p}_i|V_i|=\sum _i|F\cap V_i|=|F|,$$

$$\mathbb{E}|\partial E|=\sum _{i,j}\mathbb{P}(p_j<U\le p_i)\cdot |\partial (V_i,V_j)|=\sum _{i,j}\max \{p_i-p_j,0\}\cdot |\partial (V_i,V_j)|,$$

$$p_i-p_j\le \frac{|\partial (F\cap V_i,F^c\cap V_j)|}{|\partial (V_i,V_j)|}\text{whenever }{p}_i>p_j\text{ and }\partial (V_i,V_j)\ne \varnothing .$$

$$\mathbb{E}|\partial E|\le \sum _{i,j}|\partial (F\cap V_i,F^c\cap V_j)|=|\partial F|=\alpha \cdot \mathbb{E}|E|,$$

$$\rho(u,v):=\min\big{\{}n\geq 1:v\in\Gamma_{\mathbf{0},u}F_{n}\big{\}}.$$

$$\rho (u_i,v_i)\to \infty \text{ as }i\to \infty \text{if and only if}\mathrm{dist}(u_i,v_i)\to \infty \text{ as }i\to \infty .$$

$$|\{u\in \mathbb{V}:0\in {\Gamma }_{0,u}F\}|=|F|.$$

$$f(u,v):=1_{\{v\in {\Gamma }_{0,u}F\}}.$$

$$|\{u\in \mathbb{V}:0\in {\Gamma }_{0,u}F\}|=\sum _{u}f(u,0)=\sum _{v}f(0,v)=|{\Gamma }_{0,0}F|=|F|.\text{qed}$$

$$h(Y)=H(Y_v)=\sum _{n=1}^{\infty }H(Y_{v,n})=\sum _{n=1}^{\infty }\left({ϵ}_n\log \frac{1}{{ϵ}_n}+(1-{ϵ}_n)\log \frac{1}{1-{ϵ}_n}\right)\le 10ϵ\log \frac{1}{ϵ}.$$

$$U_n:=\{v\in \mathbb{V}:Y_{v,n}=1\},$$

$$\bar{\mathcal{C}}_{U}(u):=\Big{\{}v\in\mathbb{V}:\rho(u,v)<\rho(u^{\prime},v)\text{ for all }u^{\prime}\in U\setminus\{u\}\Big{\}},\qquad u\in U.$$

$${\mathcal{C}}_U(u):=({\bar{\mathcal{C}}}_U(u){)}^{-1}.$$

$$A_1:=\bigcup _{u\in U_1}{\mathcal{C}}_{U_1}(u).$$

$$A_{n+1}^{\prime }:=\bigcup _{u\in U_{n+1}}{\mathcal{C}}_{U_{n+1}}(u),$$

$$A_{n+1}:=A_n\cup (A_{n+1}^{\prime }\setminus D_{n+1}^{+1}).$$

$$E_{n}:=\Big{\{}B_{n-1}\subset\mathcal{C}_{U_{n}}(u)\text{ for some }u\in U_{n}\Big{\}}.$$

$$\{0\in A_n\setminus A_{n-1}\}=E_n\cap \{0\notin A_{n-1}\}.$$

Full text

1. Introduction

Consider a random process $X=(X_{v})_{v\in\mathbb{V}}$ living on the vertex set $\mathbb{V}$ of an infinite graph $G$. The process $X$ is said to be finitely dependent if its restrictions to sets which are sufficiently separated (at least some fixed distance apart) are independent. A trivial example of a finitely dependent process is a process $Y=(Y_{v})_{v\in\mathbb{V}}$ in which all random variables are independent. A natural question is then how close is a finitely dependent process to such an independent process? Before addressing this question, we first observe that “local functions” of an independent process $Y$ are always finitely dependent. That is, if $X$ is obtained from $Y$ by computing each $X_{v}$ as a function only of the random variables $Y_{u}$ for which $u$ is at a uniformly bounded distance from $v$, then $X$ is finitely dependent.

Suppose now that $G$ is transitive and henceforth restrict attention to processes $X$ which are invariant under all automorphisms of $G$ (or under a transitive subgroup of automorphisms). In particular, the independent process $Y$ considered above must now be an i.i.d. process – that is, in addition to being independent, the $\{Y_{v}\}_{v}$ are also identically distributed. If $X$ is obtained from $Y$ by applying the same local function at each vertex $v$ (i.e., the function applied at $u$ is the composition of the function applied at $v$ with any automorphism taking $u$ to $v$), then $X$ is said to be a block factor of $Y$. Thus, block factors of i.i.d. processes provide a recipe for constructing invariant finitely dependent processes.

It was a long-standing open problem [ibragimov1965independent, ibragimov1975independent] to determine whether block factors of i.i.d. processes are the only (invariant) finitely dependent processes on $\mathbb{Z}$, until finally an example was given by Burton–Goulet–Meester [burton19931] of a 1-dependent process which is not a block factor of any i.i.d. process. Recently, Holroyd and Liggett [holroyd2016finitely] showed that proper colorings distinguish between block factors of i.i.d. processes and finitely dependent processes – no proper coloring of $\mathbb{Z}$ is a block factor of an i.i.d. process, but finitely dependent proper colorings exist.

Thus, it is not true that every finitely dependent process is a block factor of an i.i.d. process. In other words, given a finitely dependent process $X$, one cannot in general hope to find an i.i.d. process $Y$ and an invariant rule for computing $X$ from $Y$, which allows to determine the value of $X_{v}$ by looking at $Y$ on a fixed-size window around $v$. The goal of this paper is to show the “next best thing” – namely, that it is possible to determine $X_{v}$ by looking at $Y$ on a variable-sized window around $v$, where the size of the window, though always finite, may vary according to the input $Y$.

We say that $X$ is a finitary factor of $Y$ if there is an invariant rule which allows to compute the value of $X$ at any vertex $v$ by only looking at variables $Y_{u}$ for which $u$ is within a certain finite, but random, distance from $v$ (formal definitions are given below). Thus, a block factor is a finitary factor in which the required distance is not only finite, but is determistically bounded by some constant. The main contribution of this paper is to prove that every finitely dependent process is a finitary factor of an i.i.d. process. This result holds on any amenable graph $G$. When it is further assumed that no two balls in $G$ with different centers are identical, it is also possible to control the entropy of the i.i.d. process involved, and the result becomes that every finitely dependent process is a finitary factor of an i.i.d. process with only slightly larger entropy.

1.1. Definitions and main result

Let $\mathbb{V}$ be a countable set, let $G$ be a graph on vertex set $\mathbb{V}$ and let $\Gamma$ be a group acting on $\mathbb{V}$. A random field (or random process) on $G$ is a collection of random variables $X=(X_{v})_{v\in\mathbb{V}}$ indexed by the vertices of $G$ and defined on a common probability space. We say that $X$ is $\Gamma$-invariant if its distribution is not affected by the action of $\Gamma$, i.e., if $(X_{\gamma v})_{v\in\mathbb{V}}$ has the same distribution as $X$ for any $\gamma\in\Gamma$. We say that $X$ is $k$-dependent if $(X_{u})_{u\in U}$ and $(X_{v})_{v\in V}$ are independent for any two sets $U,V\subset\mathbb{V}$ such that $\operatorname{dist}(u,v)>k$ for all $u\in U$ and $v\in V$. We say that $X$ is finitely dependent if it is $k$-dependent for some finite $k$.

Suppose now that $G$ is a transitive locally finite graph and that $\Gamma$ is a subgroup of the automorphism group of $G$. Let $S$ and $T$ be two measurable spaces, and let $X=(X_{v})_{v\in\mathbb{V}}$ and $Y=(Y_{v})_{v\in\mathbb{V}}$ be $S$-valued and $T$-valued $\Gamma$-invariant random fields. A coding from $Y$ to $X$ is a measurable function $\varphi\colon T^{\mathbb{V}}\to S^{\mathbb{V}}$ that is $\Gamma$-equivariant, i.e., commutes with the action of every element of $\Gamma$, and satisfies that $\varphi(Y)$ and $X$ are identical in distribution. Such a coding is also called a factor map from $Y$ to $X$, and when such a coding exists, we say that $X$ is a $\Gamma$-factor of $Y$.

Suppose now that $S$ and $T$ are countable. Let $\mathbf{0}\in\mathbb{V}$ be a distinguished vertex. The coding radius of $\varphi$ at a point $y\in T^{\mathbb{V}}$, denoted by $R(y)$, is the minimal integer $r\geq 0$ such that $\varphi(y^{\prime})_{\mathbf{0}}=\varphi(y)_{\mathbf{0}}$ for all $y^{\prime}\in T^{\mathbb{V}}$ which coincide with $y$ on the ball of radius $r$ around $\mathbf{0}$ in the graph-distance, i.e., $y^{\prime}_{v}=y_{v}$ for all $v\in\mathbb{V}$ such that $\operatorname{dist}(v,\mathbf{0})\leq r$. It may happen that no such $r$ exists, in which case, $R(y)=\infty$. Thus, associated to a coding is a random variable $R=R(Y)$ which describes the coding radius. While $S$ will always be at most countable, we will allow $T$ to be a larger space, in which case the coding radius may be similarly defined111We will only be concerned with spaces $T$ which are finite, countable or of the form $T_{1}\times T_{2}\times\cdots$ for finite sets $(T_{i})$. In the latter case, the coding radius is the smallest $r$ for which there exists $n$ such that $\varphi(y^{\prime})_{\mathbf{0}}=\varphi(y)_{\mathbf{0}}$ for all $y^{\prime}$ having the property that $y^{\prime}_{v,i}=y_{v,i}$ for all $(v,i)$ such that $\operatorname{dist}(v,\mathbf{0})\leq r$ and $1\leq i\leq n$.. A coding is called finitary if $R$ is almost surely finite. When there exists a finitary coding from $Y$ to $X$, we say that $X$ is a finitary $\Gamma$-factor of $Y$.

A graph is said to be amenable if $\inf|\partial V|/|V|=0$, where the infimum is over all finite non-empty subsets $V$ of $\mathbb{V}$, and where $\partial V$ denotes the edge-boundary of $V$.

Theorem 1.1**.**

Let $G$ be a transitive amenable graph and let $\Gamma$ be a transitive group of automorphisms of $G$. Then any finitely dependent $\Gamma$-invariant random field on $G$ is a finitary $\Gamma$-factor of an i.i.d. process.

With a minor additional constraint on the geometry of the graph $G$, we can further control the entropy of the i.i.d. process used in the coding (see Section 3.1 for the definition of entropy). The condition we require is that

[TABLE]

where $\Lambda_{r}(u)$ is the ball of radius $r$ around $u$.

Theorem 1.2**.**

Let $G$ be a transitive amenable graph satisfying (1), let $\Gamma$ be a transitive group of automorphisms of $G$, and let $X$ be a finite-valued finitely dependent $\Gamma$-invariant random field on $G$. Then for any $\epsilon>0$ there exists an i.i.d. process $Y$ with entropy $h(Y)<h(X)+\epsilon$ such that $X$ is a finitary $\Gamma$-factor of $Y$.

Let us make some remarks about how the two theorems compare to one another. In Theorem 1.2, $S$ is finite (note the assumption that $X$ is finite-valued) and, in particular, $X$ has finite entropy, while in Theorem 1.1, $S$ may be countable and $X$ may have infinite entropy. In Theorem 1.2, $Y$ has finite entropy so that $T$ is countable, whereas Theorem 1.1 may require a larger space $T$ for the conclusion to hold. In fact, in the absence of condition (1), even when $S$ is finite, it might not be possible to have $T$ countable (see Remark 4 in Section 7).

Let us also remark that, while the theorems do not assume connectivity of the graph, there is no loss of generality in assuming this, since transitivity implies that the connected components are isomorphic and finite dependence implies that the random field is independent on different components. Thus, the same coding can be used for all components.

1.2. Discussion

Finite dependence and finitary factors have applications in computer science. For example, if the graph $G$ represents machines in a network and the random field $X$ represents a common plan in which each machine $v$ is assigned a specific role $X_{v}$, then finite dependence provides certain security benefits in the face of an attacker (if someone gains access to some machines, they learn nothing about the roles of far away machines, thereby confining the security breach), and being a finitary factor of an i.i.d. process provides reliability (e.g., no single point of failure) as it means that the machines can determine their own roles in a distributed manner by following a common protocol, while using local randomness and communicating with finitely many other machines. See e.g. [linial1987distributive, naor1991lower] for more information.

The finitary coding properties of finitely dependent processes on $G=\mathbb{Z}$, and in some cases on $G=\mathbb{Z}^{d}$ with $d\geq 2$, have been studied in various contexts. We give a brief account of these works. We are unaware of any works regarding finitary factors for finitely dependent processes on other graphs.

A result by Smorodinsky [smorodinsky1992finitary] shows that every stationary (i.e., translation-invariant) finitely dependent process on $\mathbb{Z}$ is finitarily isomorphic (a stronger notion than being a finitary factor) to an i.i.d. process. This result is not for the full autormorphism group of the graph (which includes reflections), but rather only for the group of translations. In this respect, Theorem 1.2 strengthens this result (if one is content with a finitary factor, rather than a finitary isomorphism), as it provides a finitary factor which is also reflection invariant whenever the finitely dependent process is such. The proof in [smorodinsky1992finitary] is based on the so-called marker-filler method of Keane and Smorodinsky [keane1977class, keane1979bernoulli]. Unfortunately, only a brief sketch of the proof is provided in [smorodinsky1992finitary] and the details seem to be missing (after some initial steps, Smorodinsky says that the rest of the proof proceeds along the same lines as in [keane1979bernoulli] with some necessary modifications). Our proof is based on a different approach; see Section 2 for an outline.

The question of whether there exists a stationary finitely dependent process which is not a block factor of any i.i.d. process was raised by Ibragimov and Linnik [ibragimov1965independent, ibragimov1975independent] in 1965. Some progress on this question was made [aaronson1989algebraic, aaronson1992structure] until it was finally resolved in 1993 by Burton–Goulet–Meester [burton19931] who gave the first example of a stationary finitely dependent process which is not a block factor of an i.i.d. process. In fact, they showed such an example in which the finitely dependent process is a 1-dependent hidden-Markov process with finite energy. Some history about finitely dependent processes that cannot be written as block factors is given in [holroyd2016finitely].

Holroyd and Liggett [holroyd2016finitely] constructed a stationary 1-dependent 4-coloring and a stationary 2-dependent 3-coloring of $\mathbb{Z}$, neither of which is a block factor of an i.i.d. process (indeed, no coloring is such [holroyd2016finitely]). Holroyd [holroyd2017one] subsequently showed that the 1-dependent 4-coloring is a finitary factor of an i.i.d. process. Regarding the analogous statement for the 2-dependent 3-coloring, Holroyd writes in [holroyd2017one] that “one may attempt to apply our method to the 2-dependent 3-coloring, but we will see that it meets a fundamental obstacle in this case”. Our result shows that either coloring is a finitary factor of an i.i.d. process (with slightly larger entropy), answering affirmatively question (iii) in [holroyd2017one, Open problems]. In fact, the two colorings are also reflection invariant, and hence the finitary factors may also be taken to commute with reflections. Related aspects of these two colorings were studied in [holroyd2018finitely].

In a subsequent paper [holroyd2015symmetric], Holroyd and Liggett constructed, for any $q\geq 4$, a 1-dependent $q$-coloring of $\mathbb{Z}$ which is invariant under translations and reflections and is also symmetric under permutations of the colors. It was shown in [holroyd2017mallows] that each of these colorings is a finitary factor of an i.i.d. process (with exponential tail on the coding radius). Our result shows that each of these colorings is a finitary factor of an i.i.d. process, where the factor map commutes with all automorphisms of $\mathbb{Z}$ and the i.i.d. process has entropy only slightly larger than the coloring.

In [holroyd2016finitely], Holroyd and Liggett also constructed stationary finitely dependent colorings of $\mathbb{Z}^{d}$ with $d\geq 2$. More specifically, they constructed a stationary 1-dependent $4^{d}$-coloring of $\mathbb{Z}^{d}$ and a stationary finitely dependent 4-coloring of $\mathbb{Z}^{d}$. However, unlike the above one-dimensional colorings, these colorings are only translation-invariant and not automorphism-invariant. In fact, it is still unknown whether there exists a finitely dependent coloring of $\mathbb{Z}^{d}$ ($d\geq 2$) which is invariant under all automorphisms of $\mathbb{Z}^{d}$.

In the same paper [holroyd2016finitely], Holroyd and Liggett also investigated the existence of stationary finitely dependent processes on $\mathbb{Z}$ which are supported on a given shift of finite type. They showed that for any reasonably non-degenerate (namely, nonlattice) shift of finite type $\mathcal{S}$ on $\mathbb{Z}$, there exists a stationary finitely dependent process which almost surely belongs to $\mathcal{S}$. It was later shown [holroyd2017mallows] that there exists such a process which is also a finitary factor of an i.i.d. process (with exponential tail on the coding radius).

A block factor is precisely a finitary factor with bounded coding radius. Given a finitary factor which is not a block factor, it is natural to wonder about the typical value of the coding radius. As we have mentioned, the 1-dependent 4-coloring of $\mathbb{Z}$ from [holroyd2016finitely], which is not a block factor of any i.i.d. process, was shown in [holroyd2017one] to be a finitary factor of an i.i.d. process. This finitary factor was shown to have (at least) power-law tail on the coding radius, thus yielding a perhaps infinite expected coding radius. Holroyd–Hutchcroft–Levy [holroyd2017mallows] showed that there exist finitely dependent colorings of $\mathbb{Z}$ which are finitary factors of i.i.d. processes with exponential tail on the coding radius. Indeed, they showed that such a $k$-dependent $q$-coloring exists when $(k,q)$ is either $(1,5)$, $(2,4)$ or $(3,3)$. On the other hand, it is believed [holroyd2017one, holroyd2017mallows] that when $(k,q)$ is $(1,4)$ or $(2,3)$, no $k$-dependent $q$-coloring is a finitary factor of an i.i.d. process with finite expected coding radius. We mention that optimal tails for the coding radius of colorings of $\mathbb{Z}^{d}$ (which are not necessarily finitely dependent) and shifts of finite type on $\mathbb{Z}$ have been studied in [holroyd2017finitary].

Our main theorem gives no information about the coding radius beyond its almost-sure finiteness. Indeed, in light of the above discussion, it would seem that for an arbitrary finitely dependent process on $\mathbb{Z}$, there is not much hope to obtain a finitary factor with finite expected coding radius. Still, some information about the coding radius may be extracted from the proof given here (see Remark 3). For example, for 1-dependent processes on $\mathbb{Z}$, the finitary factor provided by Theorem 1.1 has a coding radius $R$ satisfying that $\mathbb{P}(R>r)\leq 8/r$ for all $r$. In the particular case of the 1-dependent 4-coloring of [holroyd2016finitely], this improves the power in the power-law bound shown in [holroyd2017one] (to an optimal power if the prediction above is indeed correct).

To the best of our knowledge, beyond Smorodinsky’s result on $\mathbb{Z}$, there do not exist any general results on the finitary coding properties of finitely dependent processes. In particular, Theorem 1.1 and Theorem 1.2 are new for any amenable graph $G$ other than $\mathbb{Z}$, and also for $G=\mathbb{Z}$ in the case when $\Gamma$ is the full automorphism group of $\mathbb{Z}$. Finally, we mention that the situation for non-amenable graphs is still poorly understood – for example, on a regular tree (of degree at least three), it is not even known whether every automorphism-invariant finitely dependent process is a (non-finitary) factor of an i.i.d. process [lyons2017factors].

1.3. Acknowledgments

I would like to thank Omer Angel, Nishant Chandgotia, Tom Meyerovitch and Mathav Murugan for useful discussions. I am especially grateful to Nishant Chandgotia for suggesting to extend the result from $\mathbb{Z}^{d}$ to transitive amenable graphs, and to Omer Angel for jointly proving Lemma 4.2 with me. I would also like to thank the referees for useful comments.

1.4. Notation

Throughout the paper, $G$ is always assumed to be an infinite, transitive, locally finite, connected graph on a countable vertex set $\mathbb{V}$, and $\Gamma$ is a subgroup of the automorphism group of $G$ that acts transitively on $\mathbb{V}$. The full automorphism group of $G$ is denoted by $\text{Aut}(G)$. The graph distance in $G$ is denoted by $\operatorname{dist}(\cdot,\cdot)$. For sets $U,V\subset\mathbb{V}$, we write $\operatorname{dist}(U,V):=\min_{u\in U,v\in V}\operatorname{dist}(u,v)$ and $\operatorname{dist}(u,V):=\operatorname{dist}(\{u\},V)$. For $r\geq 0$, denote $V^{+r}:=\{u\in\mathbb{V}:\operatorname{dist}(u,V)\leq r\}$ and $V^{-r}:=\{u\in\mathbb{V}:\operatorname{dist}(u,V^{c})>r\}$. The ball of radius $r$ around $v$ is denoted by $\Lambda_{r}(v):=\{v\}^{+r}$. The neighborhood of $V$ is $N(V):=V^{+1}\setminus V$ and the edge-boundary of $V$ is $\partial V:=\{\{u,v\}\in E(G):v\in V,u\notin V\}$.

All logarithms are taken to be in base 2 and we use the convention that $0\log 0$ is 0.

2. Outline of proof

Our goal is to express $X$, a finitely dependent invariant process, as a finitary factor of an i.i.d. process $Y$. The construction of the finitary coding involves the use of three sources of randomness: a random number of random bits located at each vertex, a so-called cell process, and a random total order on $\mathbb{V}$. The first of these three will simply be given by an i.i.d. process, denoted $Y^{\text{bits}}$, while the latter two will be obtained as finitary factors of different i.i.d. processes, denoted $Y^{\text{cell}}$ and $Y^{\text{ord}}$. In turn, $Y$ will be a triplet $Y=(Y^{\text{bits}},Y^{\text{cell}},Y^{\text{ord}})$ consisting of three mutually independent i.i.d. processes.

To illuminate the main ideas behind our construction, we provide a sketch of the proof below, explaining separately three ingredients:

1. (1)

Constructing a finitary coding: The basic and most essential part of the construction is how to obtain $X$ as a finitary factor of $Y$ when $Y$ is allowed to have infinite entropy (this is the setting of Theorem 1.1). In this case, $Y^{\text{bits}}_{v}$ and $Y^{\text{ord}}_{v}$ may be taken to be uniform random variables in $[0,1]$, and the total order may be taken to be the one induced by the usual order on $Y^{\text{ord}}_{v}$. 2. (2)

Controlling the entropy: The second part is how to control the entropy of the $Y^{\text{bits}}$ process, requiring only slightly more entropy than that of $X$. To postpone dealing with the issue of controlling the entropy of $Y^{\text{ord}}$, we shall assume in this part of the proof outline that the vertices of $G$ can be deterministically ordered in a $\Gamma$-invariant manner so that $Y^{\text{ord}}$ may be disregarded entirely (e.g., it can be taken to be a constant process). For example, the lexicographical order is such an ordering when $G$ is the graph $\mathbb{Z}^{d}$ and $\Gamma$ is the group of translations. 3. (3)

Constructing a random order: The third part is how to allow for graphs $G$ and groups $\Gamma$ which do not admit such a deterministic order. This is of course the case for general graphs, but it may also be the case for simpler graphs, such as $\mathbb{Z}$ or $\mathbb{Z}^{d}$, when $\Gamma$ is the full automorphism group of $G$. In these cases, we are led to consider random orders with suitable properties.

Already the first part above relies on the aforementioned cell process. Before introducing this process, it is convenient to observe that it suffices to prove Theorem 1.1 and Theorem 1.2 for 1-dependent random fields. To see this, let $G^{\otimes k}$ denote the graph on vertex set $\mathbb{V}$ in which two vertices are adjacent if their distance in $G$ is at most $k$. It is immediate from the definitions that $X$ is $k$-dependent as a random field on $G$ if and only if it is 1-dependent as a random field on $G^{\otimes k}$. Since any automorphism of $G$ is also an automorphism of $G^{\otimes k}$, and since $G^{\otimes k}$ is amenable and satisfies (1) whenever $G$ is such, we see that we may indeed assume that $X$ is 1-dependent. This assumption, though not at all essential, is convenient as it obviates the need to work with a different connectivity than the usual connectivity in $G$.

A cell process is a random sequence $A=(A_{1},A_{2},\dots)$ of subsets of $\mathbb{V}$ satisfying the following properties almost surely:

• •

$A_{1}\subset A_{2}\subset A_{3}\subset\cdots$.

• •

$A_{1}\cup A_{2}\cup\dots=\mathbb{V}$.

• •

For each $n\geq 1$, all connected components of $A_{n}$ are finite.

We will obtain a cell process $A$ as a finitary factor of an i.i.d. process $Y^{\text{cell}}$ with arbitrarily small entropy. We do not explain here how this is done and refer the reader to Section 4 for more details and to Figure 1 for an illustration of the construction.

(1) Constructing a finitary coding: We construct a realization of $X$ as a finitary factor of $Y$ in infinitely many steps with the idea that at the end of step $n\in\{1,2,\dots\}$, we will have defined $X$ on the region $A_{n}$. In the first step, we sample $X$ on the set $A_{1}$ – this is particularly simple as the cells in $A_{1}$ are at pairwise distance at least 2, and so, due to the 1-dependence assumption on $X$, each cell in $A_{1}$ can be sampled independently. Next, at each step $n\in\{2,3,\dots\}$, we sample $X$ on the region $A_{n}\setminus A_{n-1}$, conditioned on the value of $X$ on $A_{n-1}$, which has already been sampled in the previous steps – the key observation here is that, due again to the 1-dependence assumption on $X$, the values of $X$ on different cells in $A_{n}$ are conditionally independent – indeed, if $\{V_{i}\}_{i}$ are at pairwise distance at least 2 from one another, then for any sets $U_{i}\subset V_{i}$, given $\{X_{U_{i}}\}_{i}$, one has that $\{X_{V_{i}}\}_{i}$ are mutually conditionally independent. Since all the cells of every $A_{n}$ are finite, the above steps can be carried out in a finitary manner – that is, the conditional distribution of $X$ on a given cell depends only on the previously sampled values within that cell. Since $A_{n}$ increases to $\mathbb{V}$, the value at every given vertex will eventually be sampled, thus producing a realization of $X$ from $Y$ in a finitary and $\Gamma$-equivariant manner.

Let us be slightly more specific about the way in which we “sample $X$ on a cell”. In each cell in $A_{1}$, we distinguish a vertex by choosing the smallest element in the cell according to the order given by $Y^{\text{ord}}$. We call these distinguished vertices level 1 agents. Similarly, for each $n\geq 2$ and each cell in $A_{n}$ that is not contained in $A_{n-1}$, we select a level $n$ agent in the cell by choosing the smallest element in the cell which is not in $A_{n-1}$. Note that the level $n$ agents are obtained as a finitary factor of $(Y^{\text{cell}},Y^{\text{ord}})$. With the notion of agents, we may now say more precisely that, in step $n$ above, if $\mathcal{C}$ is a cell of $A_{n}$ that is not contained in $A_{n-1}$, then we sample $X$ on the region $\mathcal{C}\setminus A_{n-1}$ (conditionally on the previously sampled values of $X$ on $\mathcal{C}\cap A_{n-1}$) by accessing a sample of the desired distribution from the random variable $Y^{\text{bits}}_{u}$, where $u$ is the unique level $n$ agent in $\mathcal{C}$, using also the order induced by $Y^{\text{ord}}$ on $\mathcal{C}\setminus A_{n-1}$ to break any symmetries which may be present in the graph structure of this region (for example, if $G=\mathbb{Z}$ and $\Gamma$ includes reflections, then when $\mathcal{C}\setminus A_{n-1}$ is a symmetric interval around $u$, the ‘left’ and ‘right’ sides of $u$ cannot be differentiated in a $\Gamma$-equivariant way without some additional information; ordering all elements in the set is a simple way to get rid of such problems). In this interpretation, we regard $Y^{\text{bits}}_{u}$ as consisting of independent samples of $\mathbb{P}(X_{U}\in\cdot\mid X_{V}=\tau)$ for all finite $U,V\subset\mathbb{Z}^{d}$ and $\tau\in S^{V}$, most of which are never used in practice.

(2) Controlling the entropy: It is clear from the last observation above that there is plenty of waste in the above construction (in terms of the process $Y^{\text{bits}}$). The problem is that we do not know ahead of time which samples of which distributions we will need access to. The basic solution to this is to place an infinite sequence of random bits at each site, i.e., $Y^{\text{bits}}_{v}\in\{0,1\}^{\mathbb{N}}$, from which we may easily construct samples of any desired distributions (hence the name of the process $Y^{\text{bits}}$). Of course, this idea alone still does not provide any control on the entropy of $Y^{\text{bits}}$. For this, we must place a finite (perhaps random) number of random bits at each site, and somehow still be sure that we are able to construct the required samples. By a random number of random bits, we mean a random variable $W$ taking values in $\{0,1\}^{*}$, the set of finite words over $\{0,1\}$, and having the property that, conditioned on the length $|W|$ of the word, $W$ is uniformly distributed on $\{0,1\}^{|W|}$.

Suppose now that there exists a deterministic total order $\leq$ on $\mathbb{V}$ that is $\Gamma$-invariant in the sense that $u\leq v$ implies that $\gamma u\leq\gamma v$ for any $u,v\in\mathbb{V}$ and $\gamma\in\Gamma$. For instance, the lexicographical order is such an order when $G=\mathbb{Z}^{d}$ and $\Gamma$ is the translation group (but there is clearly no such order when $\Gamma$ is the full automorphism group of $\mathbb{Z}^{d}$). For the purpose of this part of the proof outline, it is convenient to further suppose that every $v\in\mathbb{V}$ has a $\leq$-successor, which we denote by $v+1$, as is the case for the lexicographical order on $\mathbb{Z}^{d}$ (in which case $v+1$ is simply $v+(1,0,\dots,0)$). The existence of such a deterministic order renders $Y^{\text{ord}}$ unneeded, allowing us to focus now only on the task of controlling the entropy of $Y^{\text{bits}}$.

The idea is to associate to each possible distribution we might require, a “simulation” which outputs a sample of the distribution in question from an input of unbiased random bits. The simulation is fed independent unbiased bits one-by-one, until at some point (a stopping time) it halts and outputs the sample. Such simulations may be done efficiently: the expected number of input bits read by the simulation is bounded by the entropy of the target distribution, up to an additive universal constant.

We shall use such simulations whenever we “sample $X$ on a cell”. If the cell $\mathcal{C}$ is large, then the entropy of $X$ on $\mathcal{C}$ is also large, and thus the above additive error is negligible. When the boundary of the cell is small in comparison to the size of the cell, the average entropy of $X$ on $\mathcal{C}$ per site will also be close to $h(X)$, the entropy of $X$ itself. Thus, it will be important that the cells in $A_{1}$ are typically large with small boundary.

This already shows that, in some sense, the average number of random bits needed to generate the samples required throughout the construction is very close to $h(X)$. However, we must place a finite number of bits at each site (more precisely, we need that $h(Y^{\text{bits}})<h(X)+\epsilon$), and even if we have more than $h(X)$ such bits at every site, it still may happen at some point during the construction that a simulation carried out by an agent $u$ requires access to many more input bits than are available in $Y^{\text{bits}}_{u}$. To solve this, we must allow to “transfer” bits from one location to another. This aspect of our construction is inspired by the algorithms in [van1999existence, harvey2006universal, spinka2018finitaryising]. The idea is that whenever an agent $u$ requires access to an additional bit (beyond those available in $Y^{\text{bits}}_{u}$), it may look for an “unused” bit at $u+1$ (the $\leq$-successor of $u$). If there are no available unused bits at $u+1$ at that time, it may then proceed to look at $u+2$, and so on.

One consequence of the above description is that the steps of the construction cannot be directly related to the levels of the cell process. That is, it will no longer be the case that after step $n$ of the construction, we will have defined $X$ on the region $A_{n}$. Instead, at any step of the construction, different regions of $G$ will be at different levels of the cell process. We will continue to use $n$ to denote the levels of the cell process, and will use $t$ to denote the step of the construction (which we henceforth also refer to as time).

The way this is done is as follows. Initially, at time $t=0$, all level 1 agents are deemed active. An active level 1 agent attempts to collect unused bits until its associated simulation halts, at which point in time the agent becomes inactive and is said to have completed level 1. Once all level 1 agents contained in some level 2 cell $\mathcal{C}$ have completed, the level 2 agent associated to $\mathcal{C}$ becomes active. An active level 2 agent proceeds in the same manner as an active level 1 agent, attempting to read bits in order to complete its associated simulation. In general, a level $n$ agent becomes active once all level $n-1$ agents contained in its associated cell have completed.

In our actual construction, it is more convenient to employ the following policy which makes the details simpler to write down: at time $t$, an agent $u$ may read at most one bit, and this bit may only be read from site $u+t$. This has the advantage that it ensures that no two agents ever try to read bits from the same location simultaneously.

(3) Constructing a random order: For a general graph $G$ and group $\Gamma$, there need not be a deterministic total order of $\mathbb{V}$ that is $\Gamma$-invariant. Instead, we construct a random total order $\leq$ on $\mathbb{V}$ whose distribution is $\Gamma$-invariant. Moreover, we construct $\leq$ as a finitary factor of an i.i.d. process $Y^{\text{ord}}$ with arbitrarily small entropy. Here finitary means that the order induced on any finite set of vertices is determined by a finite (random) subset of $\{Y^{\text{ord}}_{v}\}_{v\in\mathbb{V}}$.

In addition, the constructed order $\leq$ will have the property that its order type is almost surely the same as that of $\mathbb{Z}$. That is, almost surely, every element $v$ has a successor $v+1$ and a predecessor $v-1$, and $\{v+n\}_{n\in\mathbb{Z}}=\mathbb{V}$. Though it does not follow from the above definition of finitary, it will turn out to be the case that determining whether some vertex is the successor of some other vertex is also a finitary property (i.e., it is almost surely determined by a finite subset of $\{Y^{\text{ord}}_{v}\}_{v\in\mathbb{V}}$). Once such an order is at hand, the proof continues as outlined above.

Organization. In Section 3, we introduce some preliminaries. In Section 4, we prove the existence of a finitary cell process. In Section 5, we prove the existence of a finitary random total order having the order type of $\mathbb{Z}$. In Section 6, we give the construction of the finitary coding for the finitely dependent process $X$. Finally, we end in Section 7 with some remarks and open problems.

3. Preliminaries

Recall that $G$ is always assumed to be an infinite, transitive, locally finite, connected graph on a countable vertex set $\mathbb{V}$, and that $\Gamma$ is assumed to be a subgroup of the automorphism group of $G$ that acts transitively on $\mathbb{V}$.

3.1. Entropy

The Shannon entropy of a discrete random variable $Z$ is

[TABLE]

where the sum is taken over $z$ in the support of $Z$, or alternatively, we interpret $0\log 0$ to be 0. The measure-theoretic entropy (or Kolmogorov–Sinai entropy) of a $\Gamma$-invariant random field $X$ on an amenable graph $G$ is

[TABLE]

A Følner sequence in $G$ is a sequence $(F_{n})_{n=1}^{\infty}$ of non-empty finite subsets of $\mathbb{V}$ such that

[TABLE]

It is well-known that the entropy of $X$ may be computed along any Følner sequence:

[TABLE]

It follows that for any $\epsilon>0$ there exists $\delta>0$ such that

[TABLE]

Of course, since entropy is maximized by the uniform distribution, we also have that

[TABLE]

where $S$ is the finite set in which $X$ takes values. We note that if $Y$ is an i.i.d. process, then its entropy $h(Y)$ is equal to the entropy of its single-site distribution $H(Y_{\mathbf{0}})$.

3.2. The mass-transport principle

For $u,v\in\mathbb{V}$, denote

[TABLE]

Note that $\Gamma_{u,u}$ is the stabilizer of $u$. We say that $\Gamma$ is unimodular if

[TABLE]

It is well-known (see, e.g., [lyons2017probability, Chapter 8]) that $\Gamma$ is unimodular if and only if the following mass-transport principle holds:

[TABLE]

By diagonally $\Gamma$-invariant, we mean that $f(\gamma u,\gamma v)=f(u,v)$ for all $u,v\in\mathbb{V}$ and $\gamma\in\Gamma$. We note the well-known fact that, when $G$ is amenable, any transitive group of automorphisms $\Gamma$ is unimodular.

3.3. Simulating distributions from random bits

We shall use a result about the simulation of a given distribution from unbiased random bits. Let $\mu$ be a distribution on a countable set $\Omega$. A simulation of $\mu$ is a pair $\sf S=({\sf S}^{\text{time}},{\sf S}^{\text{out}})$ of measurable functions ${\sf S}^{\text{time}}\colon\{0,1\}^{\mathbb{N}}\to\mathbb{N}\cup\{\infty\}$ and ${\sf S}^{\text{out}}\colon\{0,1\}^{\mathbb{N}}\to\Omega$ with the properties:

• •

If $\omega$ is a sequence of independent unbiased bits, then ${\sf S}^{\text{out}}(\omega)$ has distribution $\mu$.

• •

If ${\sf S}^{\text{time}}(x)=n$ for some $x\in\{0,1\}^{\mathbb{N}}$ and $n\in\mathbb{N}$, then ${\sf S}^{\text{time}}(x^{\prime})=n$ and ${\sf S}^{\text{out}}(x^{\prime})={\sf S}^{\text{out}}(x)$ for any $x^{\prime}\in\{0,1\}^{\mathbb{N}}$ which coincides with $x$ on $\{1,\dots,n\}$.

The first property says that we can use $\sf S$ to simulate the desired distribution from random bits. The second property may be interpreted as saying that ${\sf S}^{\text{time}}$ is a stopping time and that ${\sf S}^{\text{out}}$ is adapted to the $\sigma$-algebra generated by $\{\omega_{i}:1\leq i\leq{\sf S}^{\text{time}}\}$ – that is, the simulation reads one input bit at a time, and once the stopping time is reached, the output is determined only by the bits that have already been read.

The following theorem follows from a result of Knuth and Yao [knuth1976complexity] (see Theorems 2.1 and 2.2 there and the corollary just after).

Theorem 3.1**.**

Let $Z$ be a discrete random variable. There exists a simulation $\sf S$ of $Z$ from independent unbiased bits $\omega$ satisfying that ${\sf S}^{\text{time}}(\omega)<\infty$ almost surely and $\mathbb{E}{\sf S}^{\text{time}}(\omega)\leq H(Z)+2$.

Knuth and Yao show that the above is in fact optimal in a strong sense (they provide a simulation whose stopping time is stochastically dominated by that of any other simulation). A version of this theorem was proved in [harvey2006universal, Theorem 3] via a more concrete construction. The results in [knuth1976complexity, harvey2006universal] also provide explicit exponential bounds on the probability that the simulation uses more than $n$ bits, but we shall not need this.

4. The cell process

Recall from Section 2 that a cell process is a random sequence $A=(A_{1},A_{2},\dots)$ of subsets increasing to $\mathbb{V}$ and satisfying that the connected components of each $A_{n}$ are finite. We may identify a cell process $A$ with the $\mathbb{N}$-valued random field $(\min\{n\geq 1:v\in A_{n}\})_{v\in\mathbb{V}}$. In particular, when we say that $A$ is $\Gamma$-invariant or a finitary factor, we mean that this latter process is such.

In this section, we show that finitary cell processes with arbitrarily small entropy exist on any transitive amenable graph.

Proposition 4.1**.**

Let $G$ be a transitive amenable graph and let $\epsilon>0$. There exists an i.i.d. process $Y$ of entropy at most $\epsilon$ and a cell process $A$ which is a finitary $\text{Aut}(G)$-factor of $Y$.

Let us mention that in the special case of $G=\mathbb{Z}^{d}$, several parts of the argument below can be skipped or simplified, thereby leading to a shorter proof. In the general case, certain technicalities arise which make the proof somewhat longer. The reader may therefore wish to have in mind the case of $G=\mathbb{Z}^{d}$ on a first reading.

Before giving the proof, let us explain the idea behind it; see Figure 1 for an illustration. The main idea of the construction is to use the points of a low-density Bernoulli process to construct Voronoi cells (determined from the Bernoulli process in a finitary manner), which are then used as the cells of $A_{1}$ (after slightly decreasing the Voronoi cells to ensure that they are well-separated). Using another Bernoulli process of even lower density, we again construct Voronoi cells, which are then used to obtain $A_{2}$ from $A_{1}$ by “filling in” some of the empty space between the cells of $A_{1}$, taking care not to connect cells of $A_{1}$ which are not in the same Voronoi cell (thus ensuring that an infinite cluster is not created). Repeating in this manner, we obtain an increasing sequence $A_{1}\subset A_{2}\subset\cdots$ of sets (each having only finite cells) as a finitary factor of a small entropy i.i.d. process. It will then only remain to show that $A_{n}$ increases to $\mathbb{V}$. This is where amenability comes into play.

To ensure that $A_{n}$ increases to $\mathbb{V}$, we must be careful in how we define the Voronoi cells. When the graph $G$ has a Følner sequence consisting of balls (as is the case when $G$ has subexponential growth), the Voronoi cells may be taken with respect to the graph distance in $G$. However, for the general case considered here, we must adapt the usual Voronoi cells to a certain “metric” (which is not necessarily a true metric) given by a suitable Følner sequence. Since we will need this metric to be diagonally $\Gamma$-invariant, we need to choose a Følner sequence $(F_{n})_{n}$ having the property that each $F_{n}$ is invariant under the stabilizer of some fixed vertex $\mathbf{0}\in\mathbb{V}$, i.e., $F_{n}=\Gamma_{\mathbf{0},\mathbf{0}}F_{n}$ for all $n$. It is a simple observation that any graph $G$ of subexponential growth has such a sequence for any $\Gamma$ (since there is a Følner sequence consisting of balls), and that the Cayley graph $G$ of a finitely generated amenable group $\Gamma$ also has such a sequence (since the stabilizers are trivial). As it turns out, such a Følner sequence always exists in an amenable (not necessarily transitive) graph, even when $\Gamma$ is its full automorphism group.

The following lemma (which appears to be new) shows that every amenable graph admits a Følner sequence consisting of sets which are invariant under the stabilizer of some vertex. The lemma was obtained jointly with Omer Angel.

Lemma 4.2**.**

Let $G$ be an amenable graph, let $\Gamma$ be the full automorphism group of $G$ and let $\mathbf{0}\in\mathbb{V}$. There exists a Følner sequence $(F_{n})_{n}$ in $G$ such that $F_{n}=\Gamma_{\mathbf{0},\mathbf{0}}F_{n}$ for all $n$.

Lemma 4.2 easily follows by applying the following lemma to each set of some Følner sequence, taking $\Gamma_{0}$ to be the stabilizer $\Gamma_{\mathbf{0},\mathbf{0}}$ of $\mathbf{0}$.

Lemma 4.3**.**

Let $G$ be any graph and let $\Gamma_{0}$ be a group of automorphisms of $G$ under which all orbits of $\mathbb{V}$ are finite. For any finite non-empty set $F\subset\mathbb{V}$, there exists a finite non-empty set $E\subset\mathbb{V}$ such that

[TABLE]

Proof.

We show the existence of the desired $E$ via a probabilistic method – that is, we choose a random subset $E$ of $\mathbb{V}$ and show that it satisfies the desired properties with positive probability. Let $\{V_{i}\}_{i}$ be the orbits of $\mathbb{V}$ under the action of $\Gamma_{0}$. Thus, $\{V_{i}\}_{i}$ is a partition of $\mathbb{V}$ such that each $V_{i}$ is finite (by assumption) and satisfies $V_{i}=\Gamma_{0}V_{i}$.

Let $U$ be a uniform random variable in $[0,1]$ and define

[TABLE]

Thus, each orbit $V_{i}$ is included in $E$ with probability $p_{i}$, where the choices for different $i$ are positively correlated through the use of the common variable $U$. Then

[TABLE]

and

[TABLE]

where $\partial(U,V)$ denotes the set of edges between two disjoint sets $U$ and $V$. Let us show that each term in the second sum is at most $|\partial(F\cap V_{i},F^{c}\cap V_{j})|$, i.e.,

[TABLE]

To see this, note that the right-hand side is the probability that an edge $e=\{u,v\}$ that is uniformly chosen from $\partial(V_{i},V_{j})$ belongs to $\partial(F\cap V_{i},F^{c}\cap V_{j})$, or equivalently, with the convention that $u\in V_{i}$ and $v\in V_{j}$, that $u\in F$ and $v\notin F$. Since this probability is at least $\mathbb{P}(u\in F)-\mathbb{P}(v\in F)$, it suffices to show that $u$ and $v$ are uniformly distributed in $V_{i}$ and $V_{j}$, respectively. This follows from the observation that the bipartite graph $(V_{i}\cup V_{j},\partial(V_{i},V_{j}))$ is biregular – each vertex in $V_{i}$ is adjacent to the same number of vertices in $V_{j}$, and similarly, each vertex in $V_{j}$ is adjacent to the same number of vertices in $V_{i}$. Indeed, for any $u,v\in V_{i}$ and $\gamma\in\Gamma_{0}$ such that $\gamma u=v$, the mapping $w\mapsto\gamma w$ defines a bijection between $N(u)\cap V_{j}$ and $N(v)\cap V_{j}$.

We thus conclude that

[TABLE]

where $\alpha:=|\partial F|/|F|$. Thus, $\alpha|E|-|\partial E|$ is a random variable with non-negative expectation. Since $\alpha|E|-|\partial E|$ is zero when $E$ is empty, conditioned on $E\neq\emptyset$, we still have that $\alpha|E|-|\partial E|$ has non-negative expectation. In particular, there is positive probability that $E\neq\emptyset$ and $\alpha|E|\geq|\partial E|$. Finally, since $E$ is finite and satisfies $E=\Gamma_{0}E$ almost surely, we see that $E$ satisfies the desired properties with positive probability. ∎

Suppose that $(F_{n})_{n}$ is a Følner sequence guaranteed by Lemma 4.2, i.e., $F_{n}=\Gamma_{\mathbf{0},\mathbf{0}}F_{n}$ for all $n$, and further suppose that $F_{1}\subsetneq F_{2}\subsetneq\cdots$ and $F_{1}\cup F_{2}\cup\dots=\mathbb{V}$ (there is clearly no loss in generality in doing so). Recall the definition of $\Gamma_{u,v}$ from Section 3.2. For $u,v\in\mathbb{V}$, define

[TABLE]

Using that $\Gamma_{\mathbf{0},\gamma u}=\gamma\Gamma_{\mathbf{0},u}$ for any $\gamma\in\Gamma$, one easily checks that $\rho$ is diagonally $\Gamma$-invariant, i.e., $\rho(\gamma u,\gamma v)=\rho(u,v)$ for all $\gamma\in\Gamma$. We stress that $\rho$ is not necessarily symmetric in that $\rho(u,v)$ may not equal $\rho(v,u)$. In particular, we do not claim that $\rho$ is a metric. Nevertheless, we still think of $\rho(u,v)$ as a measure of distance from $v$ to $u$. One nice property of $\rho$ that is easily verifiable and which will be important is that, for any sequence of pairs of vertices $(u_{i},v_{i})_{i=1}^{\infty}$, we have

[TABLE]

The fact that $\rho$ may not be symmetric presents a certain challenge in the proof, for which we require the following lemma to address. We note that $\rho$ is indeed symmetric when the Følner sequence $(F_{n})_{n}$ consists of balls, and that it is nearly symmetric when $G$ is a Cayley graph of $\Gamma$ in which case switching the roles of $u$ and $v$ in $\rho(u,v)$ has the same effect as replacing each $F_{n}$ with $F_{n}^{-1}$. Accordingly, in these cases, it is immediate that the two “$\rho$-balls” of radius $n$ around $\mathbf{0}$, $\{v:\rho(\mathbf{0},v)\leq n\}$ and $\{u:\rho(u,\mathbf{0})\leq n\}$, have the same size, namely $|F_{n}|$. The following lemma shows that this is in fact always true in our setting.

Lemma 4.4**.**

Let $G$ be a transitive amenable graph and let $\Gamma$ be the full automorphism group of $G$. Let $F\subset\mathbb{V}$ be invariant under the stabilizer of some vertex $\mathbf{0}$, i.e., $\Gamma_{\mathbf{0},\mathbf{0}}F=F$. Then

[TABLE]

Proof.

Define $f\colon\mathbb{V}^{2}\to[0,1]$ by

[TABLE]

Since $\Gamma_{\mathbf{0},\gamma u}=\gamma\Gamma_{\mathbf{0},u}$ for any $\gamma\in\Gamma$, it follows that $f$ is diagonally $\Gamma$-invariant. Thus, by the mass-transport principle (4),

[TABLE]

We are now ready to give the proof of Proposition 4.1.

Proof of Proposition 4.1.

Let $(\epsilon_{n})$ be a sequence to be chosen later which satisfies that $0\leq\epsilon_{n}\leq\epsilon 2^{-n}$. We shall construct a cell process $A$ as a finitary factor of the i.i.d. process $Y=(Y_{v})_{v\in\mathbb{V}}$ in which $Y_{v}=(Y_{v,n})_{n\geq 1}$ are independent random variables with $Y_{v,n}\sim\text{Ber}(\epsilon_{n})$. The entropy of $Y$ can be made arbitrary small, since

[TABLE]

As explained, the idea of the construction is to use the points in

[TABLE]

for any given $n$, to construct Voronoi cells, which are then used to define the cells of $A_{n}$. Precisely, we define the Voronoi cells of a non-empty set $U\subset\mathbb{V}$ by

[TABLE]

Thus, the Voronoi cell $\bar{\mathcal{C}}_{U}(u)$ associated to $u$ consists of all vertices $v\in\mathbb{V}$ which are closer (in the distance measured by $\rho$) to $u$ than to any other $u^{\prime}\in U$. In particular, Voronoi cells associated to different vertices in $U$ are disjoint, but they are not necessarily separated (they could be adjacent to one another). We therefore define modified Voronoi cells by slightly shrinking the sets $\bar{\mathcal{C}}_{U}(u)$. Precisely, we define

[TABLE]

Using that the Voronoi cells are disjoint, it is straightforward to check that $\operatorname{dist}(\mathcal{C}_{U}(u),\mathcal{C}_{U}(u^{\prime}))>1$ for distinct $u,u^{\prime}\in U$. We note that $\mathcal{C}_{U}(u)$ (in fact, already $\bar{\mathcal{C}}_{U}(u)$) may be empty and need not be connected.

Before proceeding with the construction of the cell process, let us first show that the Voronoi cells of $U$ are almost surely finite whenever $U$ is the set of points of an i.i.d. Bernoulli process. To this end, it suffices to show that the probability that $\bar{\mathcal{C}}_{U}(\mathbf{0})$ intersects $F_{n}\setminus F_{n-1}$ is summable over $n$. Indeed, since a fixed vertex $v\in F_{n}\setminus F_{n-1}$ belongs to $\bar{\mathcal{C}}_{U}(\mathbf{0})$ only if $U\setminus\{\mathbf{0}\}$ contains no element of $\{u:\rho(u,v)\leq n\}$, it follows from Lemma 4.4 that the probability of this is at most $p^{|F_{n}|-1}$, where $p$ is the density of the Bernoulli process. Since $|F_{n}|\geq n$, we see that $|F_{n}|\cdot p^{|F_{n}|-1}$ is summable, and hence that the $\bar{\mathcal{C}}_{U}(\mathbf{0})$ is almost surely finite.

We now turn to the construction of the cell process $A$. The first level set in the cell process is simply taken to be the vertices in a modified Voronoi cell of $U_{1}$, i.e.,

[TABLE]

Since the Voronio cells of $U_{1}$ are almost surely finite, we see that $\mathcal{C}_{U_{1}}(u)$ is almost surely finite for all $u\in U_{1}$. Since $\operatorname{dist}(\mathcal{C}_{U_{1}}(u),\mathcal{C}_{U_{1}}(u^{\prime}))>1$ for distinct $u,u^{\prime}\in U_{1}$, it follows that all connected components of $A_{1}$ are almost surely finite.

Suppose now that, for some $n\geq 1$, $A_{n}$ has been defined in such a way that all connected components of $A_{n}$ are almost surely finite, and let us now define $A_{n+1}$. Let $A^{\prime}_{n+1}$ be the union of the modified Voronoi cells of $U_{n+1}$, i.e.,

[TABLE]

and recall that (as for $A_{1}$) all connected components of $A^{\prime}_{n+1}$ are almost surely finite. Intuitively, we would like to obtain $A_{n+1}$ from $A_{n}$ by adding $A^{\prime}_{n+1}$. However, this might create infinite clusters, and we must take care to avoid this by instead only adding a suitable subset of $A^{\prime}_{n+1}$. It will suffice to slightly increase the “forbidden region” $(A^{\prime}_{n+1})^{c}$ as follows: let $D_{n+1}$ denote the union of the connected components of $A_{n}$ that intersect $(A^{\prime}_{n+1})^{c}$, and add $D_{n+1}^{+1}$ to the forbidden region. Precisely, we define

[TABLE]

It is straightforward to check that all connected components of $A_{n+1}$ are almost surely finite. Assuming that $A_{1}\cup A_{2}\cup\dots=\mathbb{V}$ almost surely, it is also easy to check using (5) that $A$ is a finitary $\text{Aut}(G)$-factor of $Y$, which would complete the proof of the proposition.

It remains only to show that $A_{1}\cup A_{2}\cup\dots=\mathbb{V}$ almost surely. By $\text{Aut}(G)$-invariance, this is equivalent to the fact that $\mathbb{P}(\mathbf{0}\in A_{n})\to 1$ as $n\to\infty$. For $n\geq 1$, let $B_{n}$ denote the connected component of $\mathbf{0}$ in $A_{n}\cup\{\mathbf{0}\}$. For $n\geq 2$, define the event

[TABLE]

Note that

[TABLE]

Thus, it suffices to show that

[TABLE]

As we now show, this holds when $\epsilon_{n}$ is suitably chosen. Since $B_{n-1}$ is almost surely finite, there exists a sufficiently large $r_{n}$ so that

[TABLE]

where $\Lambda_{r}:=\Lambda_{r}(\mathbf{0})$. Since $(F_{s})_{s}$ is a Følner sequence, there exists $s_{n}$ sufficiently large so that

[TABLE]

Set $\epsilon_{n}:=|F_{s_{n}}|^{-1}$. Then, noting that $A_{n-1}$ (and thus also $B_{n-1}$) is independent of $U_{n}$,

[TABLE]

Since the first term on the right-hand side is at least $\frac{1}{2}$ by the choice of $r_{n}$, it remains to show that, for some constant $c>0$ which does not depend on $n$, we have

[TABLE]

Let us first see how to show this when $F_{s_{n}}$ is a ball, say $\Lambda_{\ell}$. In this case, it is not hard to see that the event in question occurs when $U_{n}$ has a unique point in $\Lambda_{\ell-r_{n}}$ and no other point in $\Lambda_{\ell+r_{n}}$, so that

[TABLE]

We now handle the general case in more detail. Set $U:=U_{n}$. Our goal is to bound from below the probability that $\Lambda_{r_{n}-1}\subset\mathcal{C}_{U}(u)$ for some $u\in U$. To this end, we first find a simple condition that implies the occurrence of this event. For a set $F\subset\mathbb{V}$, denote

[TABLE]

Set $r:=r_{n}$ and $s:=s_{n}$. Let us show that

[TABLE]

Suppose that the left-hand side holds. Let us show that $\Lambda_{r-1}\subset\mathcal{C}_{U}(u)$, where $u$ is the unique element in $U\cap M(F_{s}^{-r})$. By the definition of $\mathcal{C}_{U}(u)$, this is equivalent to $\Lambda_{r}\subset\bar{\mathcal{C}}_{U}(u)$. Recalling the definition of $\bar{\mathcal{C}}_{U}(u)$, we see that we must show that $\rho(u,w)<\rho(u^{\prime},w)$ for all $u^{\prime}\in U\setminus\{u\}$ and $w\in\Lambda_{r}$. Let $u^{\prime}\in U\setminus\{u\}$ and $w\in\Lambda_{r}$. It suffices to show that $\rho(u,w)\leq s$ and $\rho(u^{\prime},w)>s$.

Towards showing this, we first note that $(\gamma V)^{+1}=\gamma(V^{+1})$ and $(\gamma V)^{-1}=\gamma(V^{-1})$ for any $\gamma\in\Gamma$ and $V\subset\mathbb{V}$, due to the fact that $\gamma$ acts by an automorphism of $G$. In particular, $(\Gamma_{\mathbf{0},u}V)^{+r}=\Gamma_{\mathbf{0},u}(V^{+r})$ and $(\Gamma_{\mathbf{0},u}V)^{-r}=\Gamma_{\mathbf{0},u}(V^{-r})$, and we may drop the parenthesis when writing such terms.

Let us now show that $\rho(u,w)\leq s$. Since $u\in M(F_{s}^{-r})$, we have that $\mathbf{0}\in\Gamma_{\mathbf{0},u}F_{s}^{-r}$, or equivalently, $\Lambda_{r}\subset\Gamma_{\mathbf{0},u}F_{s}$. Since $w\in\Lambda_{r}$, it follows that $\rho(u,w)\leq s$. Next, we show that $\rho(u^{\prime},w)>s$. Note that $u^{\prime}\notin M(F_{s}^{+r})$ since $u\in M(F_{s}^{-r})\subset M(F_{s}^{+r})$. Thus, $\mathbf{0}\notin\Gamma_{\mathbf{0},u^{\prime}}F_{s}^{+r}$, or equivalently, $\Lambda_{r}\cap\Gamma_{\mathbf{0},u^{\prime}}F_{s}=\emptyset$. Thus, $w\notin\Gamma_{\mathbf{0},u^{\prime}}F_{s}$ and, using that $F_{1},F_{2},\dots,F_{s-1}\subset F_{s}$, it follows that $\rho(u^{\prime},w)>s$.

Using (7), we obtain

[TABLE]

By Lemma 4.4, we have that

[TABLE]

Finally, by (6),

[TABLE]

so that, by standard estimates for Bernoulli random variables, both probabilities in question are bounded below by a positive constant. ∎

The above proposition established the existence of a finitary cell process $A$. In particular, $A_{n}$ is an invariant set which has high density when $n$ is large. The following proposition shows that the clusters of a dense invariant set typically have relatively small boundary.

Lemma 4.5**.**

Let $G$ be a transitive graph of degree $d$ and let $\Gamma$ be a transitive unimodular group of automorphisms of $G$. Let $B\subset\mathbb{V}$ be a random set with no infinite clusters and whose distribution is $\Gamma$-invariant. Let $\mathcal{C}_{v}$ denote the cluster of $v$ in $B$. Then, for any $\delta>0$,

[TABLE]

Proof.

The proof uses the mass-transport principle. Define $\psi\colon\mathbb{V}^{2}\to[0,1]$ by

[TABLE]

Note that, almost surely,

[TABLE]

and

[TABLE]

The $\Gamma$-invariance of $B$ implies that $f(u,v):=\mathbb{E}\psi(u,v)$ is diagonally $\Gamma$-invariant. Thus, the mass-transport principle (4) yields that

[TABLE]

The proposition now follows from Markov’s inequality. ∎

Remark 1**.**

A result of Häggström [haggstrom1997infinite, Theorem 1.6] states that any automorphism-invariant edge percolation on a $d$-regular tree ($d\geq 3$) with edge-density at least $2/d$ has an infinite cluster with positive probability. In particular, automorphism-invariant cell processes do not exist on such a tree. Moreover, by Lemma 4.5 (see also the closely related [benjamini1999group, Theorem 1.2]), we see that $\Gamma$-invariant cell processes do not exist on any transitive unimodular non-amenable graph. In fact, it is shown in [benjamini1999group, Theorem 5.1] that a closed subgroup $\Gamma$ of $\text{Aut}(G)$ is amenable if and only if there is a $\Gamma$-invariant site percolation on $G$ with with no infinite clusters and density arbitrarily close to 1. It follows that a $\Gamma$-invariant cell process on $G$ exists if and only if $\Gamma$ is amenable. The site percolation constructed in [benjamini1999group] is a factor of an i.i.d. process, though it is not finitary.

5. Random total orders

In this section, we construct a random total order on $\mathbb{V}$ that has the following properties:

• •

It is a finitary factor of an i.i.d. process with arbitrarily small entropy.

• •

It is supported on total orders having the same order type as $\mathbb{Z}$.

• •

The successor/predecessor of any vertex can be found in a finitary manner.

Let us explain these properties. A random total order on $\mathbb{V}$, or more generally, a random binary relation on $\mathbb{V}$, may be regarded as a random element in $\{0,1\}^{\mathbb{V}^{2}}$. With this viewpoint, the notion of finitary factor easily applies to such relations. Namely, such a relation is a $\Gamma$-factor of $Y$ if it has the same distribution as $\varphi(Y)$ for some measurable function $\varphi\colon T^{\mathbb{V}}\to\{0,1\}^{\mathbb{V}^{2}}$ satisfying that $\varphi(y)_{(u,v)}=\varphi(\gamma y)_{(\gamma u,\gamma v)}$ for all $\gamma\in\Gamma$, $u,v\in\mathbb{V}$ and $y\in T^{\mathbb{V}}$. Such a factor is finitary if for every $u,v\in\mathbb{V}$ there almost surely exists a finite (random) set $W\subset\mathbb{V}$ such that $\varphi(Y)_{(u,v)}$ is determined by $(Y_{w})_{w\in W}$, in the sense that $\varphi(y)_{(u,v)}=\varphi(Y)_{(u,v)}$ for any $y\in T^{\mathbb{V}}$ which coincides with $Y$ on $W$.

A total order $\leq$ on $\mathbb{V}$ has the same order type as $\mathbb{Z}$ if there is an order preserving bijection between the two ordered spaces, i.e., a bijection $f\colon\mathbb{V}\to\mathbb{Z}$ such that $f(u)\leq f(v)$ if and only if $u\leq v$. This may be equivalently formulated as saying that $\leq$ has no minimum or maximum and that there are finitely many elements between any two elements, i.e., every interval of the form $\{w\in\mathbb{V}:u\leq w\leq v\}$ is finite. In particular, in such an order, every vertex $v$ has a successor (an element $w\geq v$ such that $u\geq w$ for all $u\geq v$) and a predecessor (an element $w\leq v$ such that $u\leq w$ for all $u\leq v$).

Given a factor from $Y$ to a random total order $\leq$ on $\mathbb{V}$, we say that successors (predecessors) can be found in a finitary manner if for every $u,v\in\mathbb{V}$ there almost surely exists a finite (random) set $W\subset\mathbb{V}$ such that the event that $u$ is the $\leq$-successor ($\leq$-predecessor) of $v$ is determined by $(Y_{w})_{w\in W}$. We note that, in general, there is no direct relation to the notion of finitary factor: it may be that such a factor is finitary though successors/predecessors cannot be found in a finitary manner, or it may that successors/predecessors can be found in a finitary manner though the factor is not finitary. On the other hand, for a total order having the order type of $\mathbb{Z}$ almost surely, the second implication is easily seen to hold – if successors/predecessors can be found in a finitary manner, then the factor is necessarily finitary.

A total order which is a finitary factor of an i.i.d. process with infinite entropy is easily obtained from the order induced by uniform random variables in $[0,1]$ assigned to each vertex. It is easy to see that this order almost surely has the same order type as $\mathbb{Q}$. A total order (also with the order type of $\mathbb{Q}$) which is a finitary factor (with exponential tails on the coding radius) of an i.i.d. process with finite entropy was constructed in [harel2018finitary] for any quasi-transitive graph satisfying a geometric condition similar to (1). The application in [harel2018finitary] did not require the i.i.d. process to have arbitrarily small entropy and so this was not stated there, though it easily follows from the proof there that this is possible. Since the proof is short, we give it here. The following is essentially a reformulation of [harel2018finitary, Lemma 18] for our situation.

Lemma 5.1**.**

Let $G$ be a transitive non-empty graph satisfying (1). For any $0<\epsilon\leq\frac{1}{2}$ there exists a total order on $\mathbb{V}$ which is a finitary $\text{Aut}(G)$-factor of an i.i.d. Bernoulli process with density $\epsilon$.

Proof.

Let $\eta=(\eta_{v})_{v\in\mathbb{V}}$ be an i.i.d. Bernoulli process with density $\epsilon$. For any $v\in\mathbb{V}$, define $Z_{v}=(Z_{v,n})_{n\geq 0}\in\{0,1,\dots\}^{\{0,1,\dots\}}$ by

[TABLE]

Define a relation $\leq$ on $\mathbb{V}$ in which $u\leq v$ if and only if $Z_{u}\preceq Z_{v}$, where $\preceq$ denotes the lexicographical order on $\{0,1,\dots\}^{\{0,1,\dots\}}$. Then $\leq$ is clearly a $\text{Aut}(G)$-factor of $\eta$.

It remains to show that $\leq$ is almost surely a total order on $\mathbb{V}$ and that the factor is finitary. Since $\leq$ is clearly a preorder, to show that it is a total order, it suffices to show that $\mathbb{P}(Z_{u}=Z_{v})=0$ for distinct $u,v\in\mathbb{V}$. It then follows from the definition of the lexicographical order that the factor is finitary.

Fix $u,v\in\mathbb{V}$ distinct and consider the event

[TABLE]

Since $\mathbb{P}(E_{n})\to\mathbb{P}(Z_{u}=Z_{v})$ as $n\to\infty$, it suffices to show that $\mathbb{P}(E_{n}\mid E_{n-1})\leq 1-\epsilon$ for all $n\geq 1$. By (1) and the assumption that the graph is non-empty, we have $\Lambda_{n}(u)\setminus\Lambda_{n-1}(u)\not\subset\Lambda_{n}(v)$, as otherwise $\Lambda_{3n}(u)\subset\Lambda_{3n}(v)$, which in turn implies that $\Lambda_{3n}(u)=\Lambda_{3n}(v)$ by transitivity. Thus, there exists some $w_{n}\in\Lambda_{n}(u)\setminus(\Lambda_{n-1}(u)\cup\Lambda_{n}(v))$. Then

[TABLE]

Since $E_{n-1}$ is measurable with respect to $\eta_{\mathbb{V}\setminus\{w_{n}\}}$, it follows that $\mathbb{P}(E_{n}\mid E_{n-1})\leq 1-\epsilon$. ∎

Using Lemma 5.1 and the cell processes constructed in the previous section, we are able to construct a total order satisfying all three properties described above.

Lemma 5.2**.**

Let $G$ be a transitive amenable non-empty graph satisfying (1) and let $\epsilon>0$. Then there exists an i.i.d. process $Y$ with entropy at most $\epsilon$, and a random total order $\leq$ on $\mathbb{V}$ which almost surely has the same order type as $\mathbb{Z}$, such that $\leq$ is a finitary $\text{Aut}(G)$-factor of $Y$ for which successors/predecessors can be found in a finitary manner.

Proof.

By Lemma 5.1, there exists a total order $\preceq$ on $\mathbb{V}$ that is a finitary factor of an i.i.d. process $Y$ having entropy at most $\epsilon$. By Proposition 4.1, there exist a cell process $A$ that is a finitary factor of an i.i.d. process $Y^{\prime}$ (which we take to be independent of $Y$) having entropy at most $\epsilon$. We construct the required total order $\leq$ on $\mathbb{V}$ as a finitary factor of $(Y,Y^{\prime})$.

The idea is to use $\preceq$ to order the sites within the cells given by $A$. More precisely, we will define an increasing sequence of partial orders $\leq_{1}\,\subset\,\leq_{2}\,\subset\,\dots$ such that each $\leq_{n}$ induces a total order on every cell of $A_{n}$. Since $\bigcup_{n}A_{n}=\mathbb{V}$, this will produce a total order $\leq$ given by the union $\bigcup_{n}\leq_{n}$, which we will show has the desired properties.

Precisely, we define $\leq_{1}$ to be the relation in which $u\leq_{1}v$ whenever $u$ and $v$ belong to the same cell of $A_{1}$ and satisfy that $u\preceq v$. Then $\leq_{1}$ is clearly a partial order that induces a total order on any cell of $A_{1}$. In fact, it is the union of these total orders on the cells of $A_{1}$ (that is, it only compares vertices that belong to the same cell).

Next, suppose we have defined the partial order $\leq_{n-1}$ so that it is a union of total orders on the cells on $A_{n-1}$, and let us define $\leq_{n}$. Consider a cell $C$ of $A_{n}$ and let $D:=C\cap A_{n-1}=D_{1}\cup\dots\cup D_{k}$ be the union of the cells $D_{1},\dots,D_{n}$ of $A_{n-1}$ that are contained in $C$. We define $\leq_{n}$ in such a way that $D\leq_{n}C\setminus D$ by requiring that $u\leq_{n}v$ whenever $u\in D$ and $v\in C\setminus D$. To obtain a total order on $C$, it remains to order the vertices in $D$ and the vertices in $C\setminus D$. The latter is ordered by defining $u\leq_{n}v$ whenever $u,v\in C\setminus D$ and $u\preceq v$. The former is ordered by giving an order to the cells $D_{1},\dots,D_{k}$ and using the $\leq_{n-1}$ order within each cell – that is, we require that $\leq_{n}$ coincides with $\leq_{n-1}$ on each cell $D_{i}$, and that either $D_{i}\leq_{n}D_{j}$ or $D_{j}\leq_{n}D_{i}$ for any two cells $D_{i}$ and $D_{j}$. Finally, the order of the cells is determined by requiring that $D_{i}\leq_{n}D_{j}$ whenever $\min_{\preceq}D_{i}\preceq\min_{\preceq}D_{j}$. That is, the $i$-th cell precedes the $j$-cell in $\leq_{n}$ if and only if the $\preceq$-minimal element in the $i$-th cell is $\preceq$-smaller than the $\preceq$-minimal element in the $j$-th cell. It is straightforward that $\leq_{n}$ extends $\leq_{n-1}$ and that $\leq_{n}$ is a union of total orders on the cells of $A_{n}$.

We have thus obtained partial orders $\leq_{1},\leq_{2},\dots$ such that, for each $n$, $\leq_{n}$ extends $\leq_{n-1}$ and is a union of total orders on the cells of $A_{n}$. To show that $\leq$ has the order type of $\mathbb{Z}$, it remains to show that, almost surely, every $\leq$-interval is finite and there is no $\leq$-minimum and no $\leq$-maximum. It follows from the construction that if $u$ is the $\leq_{n}$-successor of $v$, then it is also its $\leq_{n+1}$-successor. Thus, to conclude that every $\leq$-interval is finite, it suffices to show that, for every $u,v\in\mathbb{V}$ having $u\leq v$, there exists $n$ such that $u\leq_{n}v$ and the interval $[u,v]_{\leq_{n}}$ is finite. Indeed, since $\leq_{n}$ only compares vertices within the same cell of $A_{n}$ and since all such cells are finite, all $\leq_{n}$-intervals are finite. Finally, no minimum or maximum can exist as this would contradict the invariance of $\leq$.

We have thus established that $\leq$ almost surely has the same order type as $\mathbb{Z}$. It is straightforward from the fact that the $\leq_{n}$-successor of a vertex $u$ (if it exists) is also the $\leq_{n+1}$-successor of $u$, that the constructed factor from $(Y,Y^{\prime})$ to $\leq$ has the property that successors/predecessors can be found in a finitary manner. As mentioned in the beginning of the section, this implies that the factor is also finitary. ∎

6. The finitary coding

In this section, we construct a finitary coding for finitely dependent processes. We present the details of the proof of Theorem 1.2. Theorem 1.1 may be proved in a similar manner (see Remark 2 in Section 7).

Let $X$ be a $\Gamma$-invariant finitely dependent process taking values in a finite set $S$. Recall from the proof outline in Section 2 that we shall construct a finitary factor from an i.i.d. process $Y=(Y^{\text{bits}},Y^{\text{cell}},Y^{\text{ord}})$ to $X$. Recall also that $A$ will be a cell process that is a finitary factor of $Y^{\text{cell}}$ and that $\leq$ will be a random total order on $\mathbb{V}$ that is a finitary factor of $Y^{\text{ord}}$, has the order type of $\mathbb{Z}$, and for which successors/predecessors can be found in a finitary manner. Given the cell process $A$ and the total order $\leq$, we will use the additional randomness in $Y^{\text{bits}}$ to construct a realization of $X$.

6.1. Choosing the parameters

Fix $\epsilon>0$. We shall choose the i.i.d. processes $Y^{\text{bits}},Y^{\text{cell}},Y^{\text{ord}}$ to satisfy $h(Y^{\text{bits}})<h(X)+5\epsilon$, $h(Y^{\text{cell}})\leq\epsilon$ and $h(Y^{\text{ord}})\leq\epsilon$ so that $Y$ has entropy

[TABLE]

We let $Y^{\text{bits}}$ be any i.i.d. process in which $Y^{\text{bits}}_{v}$ is a random number of random bits satisfying

[TABLE]

Recall that, by a random number of random bits, we mean a random variable $W$ taking values in $\{0,1\}^{*}$, the set of finite words over $\{0,1\}$, and having the property that, conditioned on the length $|W|$ of the word, $W$ is uniformly distributed on $\{0,1\}^{|W|}$. The desired random word can be obtained by taking $W$ to be the empty word with probability $p$ or a uniformly chosen sequence in $\{0,1\}^{m}$ with probability $1-p$, for some suitably chosen $m\geq 1$ and $0\leq p<1$. Indeed, in this case, $H(|W|)=-p\log p-(1-p)\log(1-p)$ and $\mathbb{E}|W|=pm$ so that $H(|W|)\to 0$ and $\mathbb{E}|W|\to h(X)+4\epsilon$ as $m\to\infty$ when $p=\frac{1}{m}(h(X)+4\epsilon)$. Since the entropy and length of $W$ are related via $H(W)=\mathbb{E}|W|+H(|W|)$, we see that (8) holds when $m$ is sufficiently large.

Let $\delta>0$ be as in (2). By decreasing $\delta$, we may additionally assume that

[TABLE]

Recall from Section 2 that we may assume without loss of generality that $X$ is 1-dependent. By Proposition 4.1, there exists a cell process $A$ and an i.i.d. process $Y^{\text{cell}}$ of entropy at most $\epsilon$ such that $A$ is a finitary factor of $Y^{\text{cell}}$. Since $\mathbb{P}(\mathbf{0}\in A_{n})\to 1$ as $n\to\infty$, Lemma 4.5 implies that $\mathbb{P}(|\partial A_{n}(\mathbf{0})|\geq\delta|A_{n}(\mathbf{0})|)\to 0$ as $n\to\infty$, where $A_{n}(\mathbf{0})$ is the cell of $\mathbf{0}$ in $A_{n}$. Thus, by replacing $(A_{1},A_{2},\dots)$ with $(A_{m},A_{m+1},\dots)$ for some large $m$, we may assume that $\mathbb{P}(|\partial A_{1}(\mathbf{0})|\geq\delta|A_{1}(\mathbf{0})|)$ is arbitrary small. Specifically, we require that

[TABLE]

Let $Y^{\text{ord}}$ be an i.i.d. process with entropy at most $\epsilon$ and let $\leq$ be a total order on $\mathbb{V}$ as guaranteed by Lemma 5.2 (note that if the graph $G$ contains no edges, then $X$ is already an i.i.d. process so that there is nothing to prove).

6.2. The construction of the finitary coding

Recall that the $n$-th $\leq$-successor of $v$ is denoted by $v+n$ and its $n$-th $\leq$-predecessor by $v-n$. In particular, $v\pm n$ are random elements of $\mathbb{V}$ which are determined from $Y^{\text{ord}}$ is a finitary manner. Recall also that the cell process $A$ is a finitary factor of $Y^{\text{cell}}$. It may be helpful from this point onward to think of $A$ and $\leq$ as given, and that our goal is to use them, together with the random bits of $Y^{\text{bits}}$, to construct a realization of $X$.

We shall define, for every time $t\geq 0$ and every vertex $u\in\mathbb{V}$, a random variable

[TABLE]

We shall define these inductively with the $t=0$ variables given by

[TABLE]

We think of $L^{t}_{u}$ as indicating whether $u$ read a bit at time $t$. In particular, if $L^{t}_{u}=1$ for some $t$ and $u$, then necessarily $u$ is an agent (of some level). Since at time 0 an agent looks for an available bit at its own location, (11) says that any level 1 agent reads a bit at time 0 if such a bit is available.

Before giving the main definitions of the construction, we first set up some auxiliary notation and definitions. As we have already mentioned, our construction has the property that if an agent $u$ reads a bit at some time $t$, then the bit it read is located at $u+t$, i.e., it is one of the bits of the word $Y^{\text{bits}}_{u+t}$. In particular, the total number of bits read from location $v$ by time $t$ is

[TABLE]

The bits at any location $v$ are read sequentially – the first agent to read a bit at $v$ will read $Y^{\text{bits}}_{v}(1)$, the second will read $Y^{\text{bits}}_{v}(2)$ and so on. Precisely, the bit read by $u$ at time $t$ is

[TABLE]

For this to be well-defined, we must make sure that $u$ does not try to read a non-existent bit – we mention already here that this does not occur, i.e., our definitions will ensure that

[TABLE]

The word read by $u$ by time $t$ is then

[TABLE]

where $\circ$ denotes concatenation. That is, $W^{t}_{u}$ is the word obtained by concatenating the bits read by $u$ until time $t$ in the order they were read. In particular, $W^{t}_{u}$ is a word in $\{0,1\}^{*}$ of length $|W^{t}_{u}|=L^{0}_{u}+L^{1}_{u}+\dots+L^{t}_{u}$. We emphasize that $(M^{t}_{u},W^{t}_{u})_{u\in\mathbb{V}}$ is well-defined once $(L^{i}_{u},N^{i}_{u})_{u\in\mathbb{V},0\leq i\leq t}$ is defined, as the former are functions of the latter and of $Y^{\text{bits}}$.

As explained in the proof outline, we use “simulations” to obtain samples of distributions from random bits. We first equip ourselves with simulations of all the possible distributions we may require throughout the construction of the finitary coding. The basic distributions we need are those of $X_{V}$ for a finite set $V\subset\mathbb{V}$. As we aim to obtain a $\Gamma$-equivariant factor, we must take care when dealing with random elements of $S^{V}$, as these are indexed by subsets of vertices. It would be more proper to view $X_{V}$ as a random element of $S^{|V|}$ by using the order $\leq$. Precisely, we proceed as follows. Recall the definition of a simulation from Section 3.3. By Theorem 3.1, for every ordered sequence $v_{1},\dots,v_{m}\in\mathbb{V}$ of distinct vertices, there exists a simulation ${\sf S}_{(v_{1},\dots,v_{m})}$ of $(X_{v_{1}},\dots,X_{v_{m}})\in S^{m}$ satisfying that

[TABLE]

Since the distribution of $X$ is $\Gamma$-invariant, we may suppose that ${\sf S}_{(v_{1},\dots,v_{m})}={\sf S}_{(\gamma v_{1},\dots,\gamma v_{m})}$ for all $\gamma\in\Gamma$ (e.g., by choosing a simulation for a single representative of each orbit, and then setting ${\sf S}_{(v_{1},\dots,v_{m})}$ to equal the simulation of its representative). Now, for a finite set $V\subset\mathbb{V}$, we let $v_{1},\dots,v_{m}$ be the vertices of $V$, ordered according to $\leq$, and set ${\sf S}_{V}$ to be the simulation $S_{(v_{1},\dots,v_{m})}$, where, for notational convenience, we interpret ${\sf S}^{\text{out}}_{V}$ as an element of $S^{V}$ (indexed by $V$) through the identification ${\sf S}^{\text{out}}_{V}(\omega)_{v_{i}}={\sf S}^{\text{out}}_{(v_{1},\dots,v_{m})}(\omega)_{i}$. We stress that ${\sf S}_{V}$ implicitly depends on the order $\leq$.

As we will also encounter situations in which regions of $X$ have already been sampled, we will also need simulations of the distribution of $X_{V}$ conditioned on $X_{U}$ for some finite set $U\subset\mathbb{V}$ which is disjoint from $V$. Thus, for every such $V$ and $U$ and every $\tau\in S^{U}$, we similarly let ${\sf S}_{V,U,\tau}$ be a simulation of $\mathbb{P}(X_{V}\in\cdot\mid X_{U}=\tau)$ satisfying that

[TABLE]

For ease of notation later on, we allow $V$ and $U$ to intersect and we allow $\tau$ to have any domain containing $U$, by interpreting ${\sf S}_{V,U,\tau}$ as ${\sf S}_{V\setminus U,U,\tau_{U}}$ in such a case. We also identify ${\sf S}_{V}$ with ${\sf S}_{V,\emptyset,\emptyset}$.

Recall that every cell $\mathcal{C}$ in $A_{n}$ that is not contained in $A_{n-1}$ has an associated level $n$ agent, and that this agent is “responsible” for generating the output on $\mathcal{C}\setminus A_{n-1}$. We denote by $A_{n}(v)$ the cell of $v$ in $A_{n}$, where $A_{n}(v):=\emptyset$ if $v\notin A_{n}$, by $U_{n}$ the set of level $n$ agents and, for $v\in A_{n}\setminus A_{n-1}$, by $U_{n}(v)$ the level $n$ agent associated to the cell $A_{n}(v)$.

With the above notation and definitions, we may now proceed to construct the finitary coding. Our goal is to define a random field $Z^{t}=(Z^{t}_{v})_{v\in\mathbb{V}}$, which represents the output at time $t$. This output will be a function of $(L^{i}_{u},M^{i}_{u},W^{i}_{u})_{u\in\mathbb{V},0\leq i\leq t}$ (and of course of the cell process $A$ and the total order $\leq$). Once $Z^{t}$ is defined for some $t$, it will then only remain to inductively define $(L^{t+1}_{u})_{u\in\mathbb{V}}$, as this then also defines $(M^{i}_{u},W^{i}_{u})_{u\in\mathbb{V},0\leq i\leq t+1}$ through the definitions above. This will therefore define $Z^{t+1}$ as well. We will then take a limit as $t\to\infty$ in order to obtain the output $Z=(Z_{v})_{v\in\mathbb{V}}$, which is the desired realization of $X$.

To facilitate the inductive definition of $(L^{t+1}_{u})_{u\in\mathbb{V}}$, we require some more definitions. We now regard $t\geq 0$ as fixed and suppose that $(L^{i}_{u})_{u\in\mathbb{V},0\leq i\leq t}$, and hence also $(M^{i}_{u},W^{i}_{u})_{u\in\mathbb{V},0\leq i\leq t}$, are already defined. We define two notions for a level $n$ agent: that of having reached level $n$ at time $t$, and that of having completed level $n$ of the simulation by time $t$. We define these notions inductively on $n$. We thus begin with level 1 agents. Given a level $1$ agent $u\in U_{1}$, we say that

• •

$u$ reached level $1$ by time $t$ (always, with no condition).

• •

$u$ completed level $1$ by time $t$ if the stopping time ${\sf S}^{\text{time}}_{A_{1}(u)}$ has been reached on input $W^{t}_{u,1}$.

Once a level 1 agent has completed level 1 of the simulation, the output is known on the corresponding level 1 cell of the agent. That is, if a level 1 agent $u$ completed level 1 by time $t$, then we will have

[TABLE]

To be more precise, let us define $Z^{t,1}=(Z^{t,1}_{v})_{v\in\mathbb{V}}$ by

[TABLE]

We will soon also define $Z^{t,n}=(Z^{t,n}_{v})_{v\in\mathbb{V}}$ for $n\geq 2$, with the idea that it represents the known output on all cells of level at most $n$ for which the simulation has completed. In particular, if $Z^{t,n}_{v}\neq\emptyset$ for some $v$ and $n$, then it will be the case that $Z^{t,n+1}_{v}=Z^{t,n}_{v}$. Now fix $n\geq 2$ and suppose that we have defined $Z^{t,1},\dots,Z^{t,n-1}$ and the two notions (reached and completed) for levels less than $n$, in such a way that the above property holds – namely, if a level $m\in\{1,\dots,n-1\}$ agent $u$ completed level $m$ by time $t$, then the output is known on $A_{m}(u)$ at time $t$ in the sense that $Z^{t,n-1}_{v}=Z^{t,m}_{v}\neq\emptyset$ for all $v\in A_{m}(u)$. Then, for a level $n$ agent $u\in U_{n}$, we say that

• •

$u$ reached level $n$ by time $t$ if every level $n-1$ agent $u^{\prime}\in U_{n-1}\cap A_{n}(u)$ has completed level $n-1$ by time $t$.

The idea here is that if $u$ reached level $n$ by time $t$, then the output is known on $A_{n}(u)\cap A_{n-1}$ at time $t$, and we may use this information to start generating the output on the remaining part of the cell, namely, on $A_{n}(u)\setminus A_{n-1}$. That is, we use the simulation ${\sf S}_{V,U,\tau}$ with $V=A_{n}(u)$, $U=A_{n}(u)\cap A_{n-1}$ and $\tau=Z^{t,n-1}$. We thus say that

• •

$u$ completed level $n$ by time $t$ if it reached level $n$ by time $t$ and the stopping time

[TABLE]

has been reached on input $W^{t}_{u}$.

Putting this together leads to defining $Z^{t,n}=(Z^{t,n}_{v})_{v\in\mathbb{V}}$ by

[TABLE]

The output $Z^{t}=(Z^{t}_{v})_{v\in\mathbb{V}}$ at time $t$ is then defined by $Z^{t}_{v}:=\lim_{n\to\infty}Z^{t,n}_{v}$. That is, to determine $Z^{t}_{v}$, we first look at the level $n$ at which $v$ enters the cell process, and then consider the level $n$ agent $u$ responsible for generating the output on the cell of $v$ in $A_{n}$. If $u$ has indeed completed level $n$ by time $t$, then we read the value of $Z^{t}_{v}$ from the output of the corresponding simulation.

Finally, we are ready to define $(L^{t+1}_{u})_{u\in\mathbb{V}}$. As mentioned, these numbers are always zero for non-agents, i.e., we set $L^{t+1}_{u}:=0$ for $u\notin U_{1}\cup U_{2}\cup\cdots$. Suppose now that $u\in U_{n}$ for some $n\geq 1$. We say that $u$ is active at time $t+1$ if it has reached, but has not completed, level $n$ by time $t$. Thus, if $u$ is active at time $t+1$, then ideally it would like to read a bit at that time, and indeed it may do so as long as there is an available bit at $u+t+1$ (recall that $u$ may only read a bit from location $u+t+1$ at time $t+1$). This leads us to define

[TABLE]

This completes the inductive definition of $L^{t+1}_{u}$ for all $t\geq 0$ and $u\in\mathbb{V}$.

We note that, by construction, once the output at a vertex is determined at some time, it remains unchanged at future times – that is, if $Z^{t}_{v}\neq\emptyset$ for some $v$ and $t$, then $Z^{t+1}_{v}=Z^{t}_{v}$. Denote by

[TABLE]

the time at which the output at $v$ is first determined. The output $Z=(Z_{v})_{v\in\mathbb{V}}$ is then given by

[TABLE]

This completes the construction of the finitary factor.

We record for later use the following simple property of the construction.

Lemma 6.1**.**

For any $t\geq 0$, we have that $(W^{t}_{u},Z^{t}_{u})_{u\in\mathbb{V}}$ and $(L^{t+1}_{u})_{u\in\mathbb{V}}$ are measurable with respect to $Y^{\text{cell}}$, $Y^{\text{ord}}$, $(|Y^{\text{bits}}_{v}|)_{v\in\mathbb{V}}$ and $(Y^{\text{bits}}_{v}(i))_{v\in\mathbb{V},1\leq i\leq M_{v}^{t}}$.

Proof.

The proof by induction on $t$ is straightforward from the definitions. ∎

6.3. Concluding Theorem 1.2

To conclude the proof of Theorem 1.2, we must establish two properties of the above construction: that the output it produces has the desired distribution, and that the output can be determined from $(Y^{\text{bits}},Y^{\text{cell}},Y^{\text{ord}})$ in a finitary manner. The former is stated in the following proposition whose proof is postponed to Section 6.4 below.

Proposition 6.2**.**

The output $Z$ has the same distribution as $X$.

Proof of Theorem 1.2.

The random field $Z$ is clearly a deterministic and $\Gamma$-equivariant function $\varphi$ of $(Y^{\text{bits}},Y^{\text{cell}},Y^{\text{ord}})$. Thus, in light of Proposition 6.2, we must only show that $\varphi$ is finitary. Since $T_{v}$ is almost surely finite (as $Z_{v}\neq\emptyset$ almost surely by Proposition 6.2), it suffices to show that $Z^{t}$ is finitary for every $t\geq 0$. This follows rather easily from the construction. To see this, we explain how to determine the value of $Z^{t}_{v}$ in a finitary manner.

We begin by finding the level $n$ in which $v$ enters the cell process, i.e., $v\in A_{n}\setminus A_{n-1}$, and then finding the cell $A_{n}(v)$ of $v$ in $A_{n}$. Since $A$ is a finitary factor of $Y^{\text{cell}}$, this may be done in a finitary manner. Next, we find the level $n$ agent $U_{n}(v)$ associated to the cell $A_{n}(v)$. Since this is just the $\leq$-minimal element in $A_{n}(v)\setminus A_{n-1}$, and since the order $\leq$ is a finitary factor of $Y^{\text{ord}}$, this may also be done in a finitary manner.

Let us suppose by induction that all steps of the construction up to time $t-1$ are finitary. Thus, recalling the definition of active, we see that, for any vertex $w$, we may determine in a finitary manner whether $w$ is active at time $t$. Since successors/predecessors in $\leq$ may be found in a finitary manner from $Y^{\text{ord}}$, it then also follows that $L^{t}_{w}$ may be determined in a finitary manner. Using again that successors/predecessors may be found in a finitary manner, we conclude that $W^{t}_{w}$ may be found in a finitary manner.

We would now like to check whether $u$ completed level $n$ by time $t$, and if so, find the output value. To check this, we start at level 1 and work our way up to level $n$. Thus, we first find all level $1$ agents which are contained in $A_{n}(v)$ (since the cell process and total order are finitary, this can be done in a finitary manner). Next, for each such agent $u$, we check whether $u$ completed level 1 by time $t$. Recall that the simulation ${\sf S}_{A_{1}(u)}$ depends on the cell $A_{1}(u)$ and on the order induced by $\leq$ on $A_{1}(u)$. Since the input word $W^{t}_{u}$, the cell process and the order are finitary, we see that we may determine whether $u$ completed level 1 by time $t$ in a finitary manner, and if so, also determine the output $Z^{t,1}_{w}$ for all $w\in A_{1}(u)$ in a finitary manner.

We now proceed to the next levels. Consider some level $2\leq m\leq n$. We again begin by finding all level $m$ agents which are contained in $A_{n}(v)$. For each such agent $u$, we check whether $u$ reached level $m$ by time $t$. For this we must check whether the level $m-1$ agents in $A_{m}(u)$ completed level $m-1$ by time $t$, which, by induction, may be done in a finitary manner. If $u$ reached level $m$ by time $t$, we then check whether $u$ completed level $m$ by time $t$. Similarly to before, the simulation ${\sf S}_{A_{m}(u),A_{m}(u)\cap A_{m-1},Z^{t,m-1}}$ depends on $A_{m}(u)$, $A_{m}(u)\setminus A_{m-1}$, the order induced by $\leq$ on $A_{m}(u)$, and on $(Z^{t,m-1}_{w})_{w\in A_{m}(u)\cap A_{m-1}}$. Since the input word $W^{t}_{u}$, the cell process and the order are finitary, we see that we may determine whether $u$ completed level $m$ by time $t$ in a finitary manner, and if so, also determine the output $Z^{t,m}_{w}$ for all $w\in A_{m}(u)$ in a finitary manner.

Continuing up to level $m=n$ yields that $Z^{t,n}_{v}$ may be determined in a finitary manner. Since $n$ is the level in which $v$ enters the cell process, we have by definition that $Z^{t}_{v}=Z^{t,n}_{v}$. Thus, $Z^{t}_{v}$ may be determined in a finitary manner, as required. ∎

6.4. The output has the correct distribution

In this section, we prove Proposition 6.2. The proof is split up into several steps. The first step is the following lemma which formalizes the intuition that the simulations used in the construction are ‘fed’ independent unbiased bits.

Lemma 6.3**.**

Let $\omega\in\{0,1\}^{\mathbb{N}}$ consist of a sequence of independent unbiased bits. Let $(\omega_{u})_{u\in\mathbb{V}}$ be a collection of i.i.d. copies of $\omega$, independent of $(Y^{\text{bits}},Y^{\text{cell}},Y^{\text{ord}})$. Then, for any $t\geq 0$, conditioned on $(Y^{\text{cell}},Y^{\text{ord}})$, the collection $(W^{t}_{u}\circ\omega_{u})_{u\in\mathbb{V}}$ has the same distribution as $(\omega_{u})_{u\in\mathbb{V}}$.

Proof.

We prove the statement by induction on $t$, taking $t=-1$ as a trivial base case (where $W^{-1}_{u}:=\emptyset$ for all $u\in\mathbb{V}$). Suppose now that we know it for some $t\geq-1$ and let us show it for $t+1$. Recall that $W^{t+1}_{u}=W^{t}_{u}\circ\hat{W}^{t+1}_{u}$. Thus, we need to show that, conditioned on $(Y^{\text{cell}},Y^{\text{ord}})$, the collection $(W^{t}_{u}\circ\hat{W}^{t+1}_{u}\circ\omega_{u})_{u\in\mathbb{V}}$ has the same distribution as $(\omega_{u})_{u\in\mathbb{V}}$. To this end, it suffices to show that, conditioned on $(Y^{\text{cell}},Y^{\text{ord}})$, the collections $(W^{t}_{u})_{u\in\mathbb{V}}$ and $(\hat{W}^{t+1}_{u}\circ\omega_{u})_{u\in\mathbb{V}}$ are independent and that the conditional distribution of the latter is that of $(\omega_{u})_{u\in\mathbb{V}}$. Indeed, the induction hypothesis will then yield the desired result.

We may restate our goal as showing that, conditioned on $(Y^{\text{cell}},Y^{\text{ord}})$ and $(W^{t}_{u})_{u\in\mathbb{V}}$, the collection $(\hat{W}^{t+1}_{u}\circ\omega_{u})_{u\in\mathbb{V}}$ has the same distribution as $(\omega_{u})_{u\in\mathbb{V}}$. Let $\mathcal{F}$ be the $\sigma$-algebra generated by $Y^{\text{cell}}$, $Y^{\text{ord}}$, $(|Y^{\text{bits}}_{v}|)_{v\in\mathbb{V}}$ and $(Y^{\text{bits}}_{v}(i))_{v\in\mathbb{V},1\leq i\leq M^{t}_{v}}$. By Lemma 6.1,

[TABLE]

are $\mathcal{F}$-measurable. Since $(\omega_{u})_{u\in\mathbb{V}}$ is independent of $(Y^{\text{bits}},Y^{\text{cell}},Y^{\text{ord}})$ and hence also of $\mathcal{F}$, it suffices to show that, conditioned on $\mathcal{F}$, the random variables $(\hat{W}^{t+1}_{u})_{u\in Q}$ are independent unbiased bits.

Note that $Q$ is the set of vertices (agents) that read a bit at time $t+1$, that

[TABLE]

is the set of vertices from which a bit was read at time $t+1$, and that $u\mapsto u+t+1$ defines a $\mathcal{F}$-measurable bijection from $Q$ to $Q^{\prime}$. Recall also that $\hat{W}^{t+1}_{u}=Y^{\text{bits}}_{u+t+1}(M^{t+1}_{u+t+1})$ for $u\in Q$. Thus, it suffices to show that, conditioned on $\mathcal{F}$, the random variables $(Y^{\text{bits}}_{v}(M^{t+1}_{v}))_{v\in Q^{\prime}}$ are independent unbiased bits.

Indeed, since $Q^{\prime}$ and $(M^{t+1}_{v})_{v\in Q^{\prime}}$ are $\mathcal{F}$-measurable by Lemma 6.1, since $M^{t+1}_{v}>M^{t}_{v}$ for all $v\in Q^{\prime}$, and since $Y^{\text{bits}}$ is an i.i.d. process that is independent of $(Y^{\text{cell}},Y^{\text{ord}})$, we see that the random variables $(Y^{\text{bits}}_{v}(M^{t+1}_{v}))_{v\in Q^{\prime}}$ are conditionally independent given $\mathcal{F}$, and that, for any $v\in Q^{\prime}$, the conditional distribution of $Y^{\text{bits}}_{v}(M^{t+1}_{v})$ is the same as the distribution of $Y^{\text{bits}}_{v}(M^{t+1}_{v})$ given $|Y^{\text{bits}}_{v}|$. Since $Y^{\text{bits}}_{v}$ is a random number of random bits, the latter is the distribution of an unbiased bit, and the proof is complete. ∎

We will use the above lemma for fixed $t$ and then let $t$ tend to infinity. In doing so, we will encounter the limiting word $W^{\infty}_{u}:=\lim_{t\to\infty}W^{t}_{u}$. Since $W^{t+1}_{u}$ extends $W^{t}_{u}$, this limit is well-defined and is a word in $\{0,1\}^{*}$ or $\{0,1\}^{\mathbb{N}}$ (we will see that it is in fact a finite word almost surely).

The next step towards proving Proposition 6.2 is to show that the output at every vertex $v$ is eventually determined, i.e., that $Z_{v}\neq\emptyset$ (equivalently, $T_{v}<\infty$) almost surely. For this, we first show that every vertex is eventually inactive.

Lemma 6.4**.**

Every vertex is almost surely eventually inactive. That is, for any $u\in\mathbb{V}$, there almost surely exists a finite $t_{0}$ such that $u$ is not active at any time $t\geq t_{0}$.

Proof.

Define

[TABLE]

Note that $\psi(u,v)$ indicates whether $u$ read a bit located at $v$. Since an agent may read at most one bit from any location, $\psi(u,v)$ also represents the number of bits read by $u$ from location $v$. Thus, recalling that $|W^{t}_{u}|=L^{0}_{u}+L^{1}_{u}+\dots+L^{t}_{u}$, we have

[TABLE]

The left-hand side describes the number of bits read by a given site $u$, while the right-hand side describes the number of bits read from a given site $v$. The mass-transport principle (4) tells us that these quantities are the same in expectation:

[TABLE]

Let $E_{u}$ be the event that $u$ is active at infinitely many times $t$. We wish to show that $\mathbb{P}(E_{u})=0$. Note that, by (13), the event $E_{u}$ is contained in the event that for all but finitely many $t\geq 0$, all bits at location $u+t$ have been read by time $t$, i.e.,

[TABLE]

where the equality follows from the fact that $M^{t}_{v}\leq M^{\infty}_{v}\leq|Y^{\text{bits}}_{v}|$ for all $v\in\mathbb{V}$ and $t\geq 0$. Suppose now that $\mathbb{P}(E_{u})>0$. Then by ergodicity, almost surely, $E_{w}$ occurs for some $w\in\mathbb{V}$, and in particular, there almost surely exists $w\in\mathbb{V}$ such that $M^{\infty}_{w+i}=|Y^{\text{bits}}_{w+i}|$ for all $i\geq 0$. Since $\{w+i\}_{i\in\mathbb{Z}}=\mathbb{V}$ almost surely, it follows by $\Gamma$-invariance that $M^{\infty}_{v}=|Y^{\text{bits}}_{v}|$ for all $v\in\mathbb{V}$ almost surely. Thus, by (14),

[TABLE]

That is, the expected number of bits read by each site is precisely the expected number of available bits per site. It remains to show that this is impossible.

Define

[TABLE]

Recall that $U_{n}(v)$ is the level $n$ agent associated to the cell $A_{n}(v)$. Since a level $n$ agent $u$ is responsible for simulating the output on $A_{n}(u)\setminus A_{n-1}$ and does so via the input word $W^{\infty}_{u}$, we may think of $\phi(u,v)$ as follows: every level $n$ agent $u$ equally divides a total ‘cost’ of $|W^{\infty}_{u}|$ among the vertices it ‘serviced’. Observe that

[TABLE]

where $N_{v}$ is the level at which $v$ entered the cell process, i.e., $v\in A_{N_{v}}\setminus A_{N_{v}-1}$, and where we used that $A_{n}(v)=A_{n}(u)$ whenever $U_{n}(v)=u$. Thus, by (15) and the mass-transport principle (4),

[TABLE]

This relates the expected number of available bits per site to the length of the input words used by the simulations. We would like to reach a contradiction to the fact that there are many available bits and that the simulation is efficient.

Suppose that $u$ is a level $n$ agent. It is straightforward from the definitions that the stopping time ${\sf S}^{\text{time}}_{A_{n}(u),A_{n}(u)\cap A_{n-1},Z^{t,n-1}}$ is not reached on any prefix of $W^{t}_{u}$ that is not $W^{t}_{u}$ itself (it may or may not be reached on the entire word $W^{t}_{u}$). It therefore follows from Lemma 6.3 that, conditioned on $(Y^{\text{cell}},Y^{\text{ord}})$,

[TABLE]

where $\omega\in\{0,1\}^{\mathbb{N}}$ consists of a sequence of independent unbiased bits, independent of $Y$. Note that, if $u$ reached level $n$ by time $t$, then $Z^{t,n-1}$ coincides with $Z$ on $A_{n}(u)\cap A_{n-1}$. Thus, taking expectations and $t\to\infty$, we obtain that

[TABLE]

Hence, by (12) and (3), on the event that $u\in U_{n}$, we have

[TABLE]

where we denote $H_{X}(V):=H(X_{V})$ for a finite set $V\subset\mathbb{V}$. Therefore, by (2) and the choice of $\delta$,

[TABLE]

where $E$ is the event that $N_{v}=1$ and $|\partial A_{1}(v)|\leq\delta|A_{1}(v)|$. Thus, by (16),

[TABLE]

Using (9) and (10), we see that $\mathbb{E}|Y^{\text{bits}}_{v}|<h(X)+3\epsilon$, which contradicts (8). We therefore conclude that $\mathbb{P}(E_{u})=0$ as required. ∎

We are now ready to show that the output at every vertex is eventually determined.

Lemma 6.5**.**

For any $v\in\mathbb{V}$, we have that $T_{v}<\infty$ almost surely.

Proof.

Since $A_{n}$ almost surely increases to $\mathbb{V}$, it suffices to show that $\mathbb{P}(Z_{v}=\emptyset\text{ and }v\in A_{n})=0$ for all $n\geq 1$. We prove this by induction on $n$, taking $n=0$ as a trivial base case by setting $A_{0}:=\emptyset$. Let $n\geq 1$ and suppose that $\mathbb{P}(Z_{v}=\emptyset\text{ and }v\in A_{n-1})=0$. By $\Gamma$-invariance, we actually have that $\mathbb{P}(Z_{w}=\emptyset\text{ for some }w\in A_{n-1})=0$. We may thus assume that $Z_{w}\neq\emptyset$ for all $w\in A_{n-1}$. Suppose now that $Z_{v}=\emptyset$ and $v\in A_{n}$. Let $u$ be the level $n$ agent $U_{n}(v)$ associated to the cell $A_{n}(v)$. Observe that, by the definition of $Z_{v}$ and $Z^{t,n}_{v}$, we have that, for all $t\geq 0$, $u$ did not complete level $n$ by time $t$. On the other hand, since $Z_{w}\neq\emptyset$ for all $w\in A_{n-1}$, there exists a finite $t_{0}\geq 0$ such that $u$ has reached level $n$ by time $t_{0}$. It follows that $u$ is active at time $t$ for every $t>t_{0}$. By Lemma 6.4, almost surely, no vertex is active at infinitely many times, thus completing the proof that $\mathbb{P}(Z_{v}=\emptyset\text{ and }v\in A_{n})=0$. ∎

Now that we have established that the output at every vertex is eventually determined, it remains to show that the distribution of the output is the correct one, namely, that of $X$. The following immediately implies Proposition 6.2.

Proposition 6.6**.**

Conditioned on $(Y^{\text{cell}},Y^{\text{ord}})$, $Z$ almost surely has the same distribution as $X$, where we regard $X$ as independent of $(Y^{\text{cell}},Y^{\text{ord}})$.

Proof.

Throughout the proof, we regard $X$ as independent of $Y^{\text{cell}}$ and $Y^{\text{ord}}$. We also condition on $(Y^{\text{cell}},Y^{\text{ord}})$ throughout the entire proof, without explicitly mentioning this. In particular, any statement about distributions or independence should be understood as conditional on $(Y^{\text{cell}},Y^{\text{ord}})$.

Since every finite subset of $\mathbb{V}$ is almost surely contained in some cell of the cell process, it suffices to show that, for any $n\geq 1$ and any cell $\mathcal{C}$ of $A_{n}$, $Z_{\mathcal{C}}$ has the same distribution as $X_{\mathcal{C}}$. We prove this by induction on $n$, taking $n=0$ as a trivial base case (where $A_{0}:=\emptyset$).

Suppose now that $n\geq 1$. Let $\mathcal{C}$ be a cell of $A_{n}$ and denote $\mathcal{C}^{\prime}:=\mathcal{C}\cap A_{n-1}$. We will show that

[TABLE]

and

[TABLE]

By feasible $\tau$, we mean that $\mathbb{P}(X_{\mathcal{C}^{\prime}}=\tau)>0$. The desired equality in distribution $Z_{\mathcal{C}}\,{\buildrel d\over{=}}\,X_{\mathcal{C}}$ follows immediately from (17) and (18). Both parts require some type of independence, which we now establish.

Let $\omega\in\{0,1\}^{\mathbb{N}}$ consist of a sequence of independent unbiased bits. Let $(\omega_{u})_{u\in\mathbb{V}}$ be a collection of i.i.d. copies of $\omega$, independent of $Y^{\text{bits}}$. By Lemma 6.3, for any $t\geq 0$, $(W^{t}_{u}\circ\omega_{u})_{u}$ has the same distribution as $(\omega_{u})_{u}$. Taking the limit as $t\to\infty$, we see that $(W^{\infty}_{u}\circ\omega_{u})_{u}$ also has the same distribution as $(\omega_{u})_{u}$.

Observe that, by construction, if $C$ is some cell of the cell process, then $Z^{t}_{C}$ is a function of $(W^{t}_{u})_{u\in C}$. Taking the limit as $t\to\infty$, it follows that $Z_{C}$ is a function of $(W^{\infty}_{u})_{u\in C}$. It also follows from the definition of $Z$ and the fact that $T_{v}<\infty$ for all $v$, that $Z$ is unchanged by concatenating any word to any $W^{\infty}_{u}$. In particular, $Z_{C}$ is also a function of $(W^{\infty}_{u}\circ\omega_{u})_{u\in C}$.

We now show (17). To this end, let $\mathcal{C}_{1},\dots,\mathcal{C}_{m}$ be the cells in $A_{n-1}$ that are contained in $\mathcal{C}$, so that $\mathcal{C}^{\prime}=\mathcal{C}_{1}\cup\dots\cup\mathcal{C}_{m}$. By the induction hypothesis, $Z_{\mathcal{C}_{j}}\,{\buildrel d\over{=}}\,X_{\mathcal{C}_{j}}$ for every $1\leq j\leq m$. Since $A$ is a cell process, we have that $\operatorname{dist}(\mathcal{C}_{j},\mathcal{C}_{j^{\prime}})>1$ for $1\leq j<j^{\prime}\leq m$. Hence, using that $X$ is 1-dependent, we see that $\{X_{\mathcal{C}_{j}}\}_{1\leq j\leq m}$ are independent. Thus, it remains to show that $\{Z_{\mathcal{C}_{j}}\}_{1\leq j\leq m}$ are also independent. Since $Z_{\mathcal{C}_{j}}$ is a function of $(W^{\infty}_{u}\circ\omega_{u})_{u\in\mathcal{C}_{j}}$, this follows from the fact that $\{W^{\infty}_{u}\circ\omega_{u}\}_{u\in\mathbb{V}}$ are independent.

To complete the proof, it remains to show (18). Let $w$ be the agent associated to $\mathcal{C}$ and recall that $w\in\mathcal{C}\setminus\mathcal{C}^{\prime}$ and that $\mathcal{C}=A_{n}(w)$. Note that

[TABLE]

Recall that ${\sf S}_{\mathcal{C},\mathcal{C}^{\prime},Z}$ is shorthand for ${\sf S}_{\mathcal{C}\setminus\mathcal{C}^{\prime},\mathcal{C}^{\prime},Z_{\mathcal{C}^{\prime}}}$. Since $W^{\infty}_{w}\circ\omega_{w}$ is independent of $(W^{\infty}_{u}\circ\omega_{u})_{u\neq w}$, and hence also of $Z_{\mathcal{C}^{\prime}}$, we conclude that the conditional distribution of $Z_{\mathcal{C}\setminus\mathcal{C}^{\prime}}$ given that $Z_{\mathcal{C}^{\prime}}=\tau$ is equal to the distribution of ${\sf S}^{\text{out}}_{\mathcal{C}\setminus\mathcal{C}^{\prime},\mathcal{C}^{\prime},\tau}(W^{\infty}_{w}\circ\omega_{w})$. Thus, using that $W^{\infty}_{w}\circ\omega_{w}$ has the same distribution as $\omega$, we see that the distribution in question is that of ${\sf S}^{\text{out}}_{\mathcal{C}\setminus\mathcal{C}^{\prime},\mathcal{C}^{\prime},\tau}(\omega)$, which is by definition $\mathbb{P}(X_{\mathcal{C}\setminus\mathcal{C}^{\prime}}\in\cdot\mid X_{\mathcal{C}^{\prime}}=\tau)$, as required. ∎

7. Remarks and open problems

Remark 2**.**

We have given the details of the proof of Theorem 1.2. Theorem 1.1 follows the same lines of proof, with minor modifications, all of which are in fact simplifications.

To obtain a proof of Theorem 1.1 with the least modifications to the existing proof, we may replace the random total order constructed in Lemma 5.1 with the total order induced by an i.i.d. process consisting of uniform $[0,1]$ random variables. Using this order in the proof of Lemma 5.2 yields a random total order with the same properties as in Lemma 5.2 (except for the bound on the entropy of the i.i.d. process). When $X$ is finite-valued, the proof then goes through with no further modifications. Otherwise, we let $Y^{\text{bits}}_{v}$ consist of infinitely many independent random bits, and then the proof goes through after an additional modification to the proof of Lemma 6.4 (which relied on the fact that the entropy of $X$ is finite to deduce a bound on the expected number of bits used by the simulation; instead we only rely on the fact that, almost surely, the simulation uses only finitely many bits; see Theorem 3.1).

It is instructive to note that a shorter and conceptually simpler proof exists when one does not need to worry about the entropy of the i.i.d. process. This is essentially what is described in “constructing a finitary coding” in Section 2. One way to implement the described coding would be to simply replace $u\pm t$ with $u$ everywhere in the construction in Section 6. That is, instead of having an agent $u$ try to read an unused bit from location $u+t$ at time $t$, it always reads bits located at $u$. Since we may place an infinite sequence of bits at every vertex, it will never run out of available bits. In this way, there is no “moving around” of bits from one location to another. This could be made conceptually even simpler if instead of using simulations from random bits to obtain samples of distributions as they are needed, from the start, each $Y^{\text{bits}}_{v}$ is a collection $(W_{V,U,\tau})_{V,U,\tau}$ of independent random variables having distribution $\mathbb{P}(X_{V}\in\cdot\mid X_{U}=\tau)$ for all finite $U,V\subset\mathbb{V}$ and $\tau\in S^{U}$. Either way, a nice feature of this construction is that the coding radius depends only on the cell process $A$ constructed in Section 4. Namely, the coding radius for determining $X_{\mathbf{0}}$ is at most the maximum of $\min\{r\geq 0:A(\mathbf{0})\subset\Lambda_{r}(\mathbf{0}\}\}$, where $A(\mathbf{0})$ is the cell of $\mathbf{0}$ in $A_{\min\{n\geq 1:\mathbf{0}\in A_{n}\}}$, the coding radius for determining the cell $A(\mathbf{0})$ and the coding radius for determining the cell process on $A(\mathbf{0})$, i.e., $(A_{n}\cap A(\mathbf{0}))_{n\geq 1}$. We elaborate on this in the next remark.

Remark 3**.**

Our main theorems give no information about the coding radius beyond its almost-sure finiteness. However, some information about the coding radius may be extracted from the proof given here. Specifically, Theorem 1.1 may be enhanced to give a universal bound on the tail of the coding radius for any fixed graph and finite-dependence range. More precisely, for any transitive amenable graph $G$ and any integer $k\geq 1$, there exists a sequence $(c_{n})_{n=1}^{\infty}$ tending to zero such that any $k$-dependent invariant random field $X$ on $G$ is a finitary factor of an i.i.d. process with a coding radius $R$ satisfying that $\mathbb{P}(R\geq n)\leq c_{n}$ for all $n$. The sequence $(c_{n})$ depends only on the graph $G$ and on the parameter $k$, and not on the group $\Gamma$ nor on the random field $X$. Indeed, the sequence $(c_{n})$ is governed by the properties of the cell process (see the last part of the previous remark). In particular, for many concrete choices of $G$ (and $k$), an explicit sequence $(c_{n})$ may be found.

To illustrate this in a simple setting, let us show that for $G=\mathbb{Z}$ and $k=1$, one may take $c_{n}=8/n$. In this case, instead of using the construction given in Section 4, it is simpler to consider the finitary cell process $A$ given by $A_{n}:=B_{1}\cup\dots\cup B_{n}$, where $(B_{n})_{n\geq 1}$ are independent random subsets of $\mathbb{Z}$, each being an independent Bernoulli percolation with parameter $1/2$. Then the level $N:=\min\{n:0\in A_{n}\}$ at which [math] enters the cell process is a geometric random variable with parameter $1/2$, conditioned on which, the lengths $L^{\pm}:=\min\{m\geq 1:\pm m\notin A_{N}\}$ of the cell of [math] in $A_{N}$ in the positive/negative directions are (independent) geometric random variables with parameter $2^{-N}$, and the coding radius $R$ for determining $X_{0}$ is bounded by $\max\{L^{+},L^{-}\}$. Thus,

[TABLE]

where we used the substitution $n=\lfloor\log_{2}r\rfloor-m$.

We remark that if one would like to simultaneously control also the entropy of the i.i.d. process (as in Theorem 1.2), then it is plausible that this can be done by allowing $(c_{n})$ to depend on the entropy gap $\epsilon$ (and perhaps on $|S|$), but we did not pursue this.

Remark 4**.**

We do not know whether condition (1) is necessary as stated in Theorem 1.2, however, as we now explain, some condition of this form is needed (i.e., the condition cannot be completely dropped). Let $G$ be an infinite transitive graph on vertex set $\mathbb{V}$ and let $H$ be a finite transitive graph on $m\geq 2$ vertices. Let $G^{\prime}$ be the graph obtained by replacing each vertex of $G$ with a copy of $H$ – that is, the vertex set of $G^{\prime}$ is $\mathbb{V}\times\{1,\dots,m\}$, and $(u,i)$ and $(v,j)$ are adjacent in $G^{\prime}$ if and only if $u$ and $v$ are adjacent in $G$, or $u=v$ and $i$ and $j$ are adjacent in $H$. Any graph $G^{\prime}$ obtained in this manner is transitive, but fails to satisfy (1). Indeed, the balls of radius 2 (or even 1 when $H$ is a complete graph) around $(v,i)$ and $(v,j)$ coincide. A simple case to have in mind is when $G=\mathbb{Z}$ and $H$ consists of an edge on two vertices, so that the vertices of $G^{\prime}$ are $\mathbb{Z}\times\{0,1\}$ and there is an edge between $(u,i)$ and $(v,j)$ if and only if $|u-v|\leq 1$.

Let $G^{\prime}$ be any graph as above and let $(W_{v})_{v\in\mathbb{V}}$ be independent uniform random variables on $\{1,\dots,m\}$. Consider the random field $X$ on $G^{\prime}$ defined by $X_{(v,i)}:=\mathbf{1}_{\{W_{v}=i\}}$. It is clear that $X$ is 2-dependent and $\text{Aut}(G^{\prime})$-invariant. We claim that $X$ is not a $\text{Aut}(G^{\prime})$-factor of any i.i.d. process $Y$ on $G^{\prime}$ whose single-site distribution has at least one atom (in particular, $Y$ cannot have finite entropy). Indeed, for any such process $Y$, the event $Y_{(v,1)}=\dots=Y_{(v,m)}$ has positive probability, and on this event there is no $\text{Aut}(G^{\prime})$-equivariant way to distinguish between $(v,1),\dots,(v,m)$. That is, any $\text{Aut}(G^{\prime})$-equivariant function $\varphi\colon T^{\mathbb{V}\times\{1,\dots,m\}}\to\{0,1\}^{\mathbb{V}\times\{1,\dots,m\}}$ must satisfy $\varphi(y)_{(v,1)}=\dots=\varphi(y)_{(v,m)}$ whenever $y\in T^{\mathbb{V}\times\{1,\dots,m\}}$ is such that $y_{(v,1)}=\dots=y_{(v,m)}$. In particular, the event $\varphi(Y)_{(v,1)}=\dots=\varphi(Y)_{(v,m)}$ has positive probability, and hence, $\varphi(Y)$ cannot have the same distribution as $X$.

We remark that there are transitive subgroups $\Gamma$ of $\text{Aut}(G^{\prime})$ for which the above obstruction does not exist. For example, let $\Gamma$ be the subgroup of $\text{Aut}(G^{\prime})$ generated by $\text{Aut}(G)$ and $\text{Aut}(H)$, both of which are naturally embedded in $\text{Aut}(G^{\prime})$. Simple modifications to the proofs of Lemma 5.1 and Lemma 5.2 yield a $\Gamma$-invariant random total order with the desired properties. The rest of the proof then goes through unchanged showing that our main result holds in this case: any finitely dependent $\Gamma$-invariant process on $G^{\prime}$ is a finitary $\Gamma$-factor of an i.i.d. process with slightly larger entropy. We believe that our main result holds in many similar situations, where condition (1) is replaced by a suitable condition on $\Gamma$. We did not pursue this direction.

Open problems.

1. (1)

One may wonder about the situation on non-amenable graphs such as a regular tree (of degree at least three). Namely, is every automorphism-invariant finitely dependent process on a tree a finitary factor of an i.i.d. process? In fact, even the more fundamental question of whether such a process is a factor of i.i.d. (without the finitary condition) is still open; see [lyons2017factors, Question 2.2]. The same questions may be asked on any transitive non-amenable graph. 2. (2)

Does there exist a stationary finitely dependent process on $\mathbb{Z}$ (or, more generally, on some transitive amenable graph) that cannot be expressed as a finitary factor of an i.i.d. process with finite expected coding radius? As mentioned, the 1-dependent 4-coloring and 2-dependent 3-coloring of [holroyd2016finitely] are believed to be examples of such processes, but this is still unproved. 3. (3)

A finitary isomorphism is a finitary factor that is invertible and whose inverse is also finitary. Somorodinky [smorodinsky1992finitary] showed that every stationary finitely dependent process on $\mathbb{Z}$ is finitarily isomorphic to an i.i.d. process. Is this true in higher dimensions? Namely, is every stationary finitely dependent process on $\mathbb{Z}^{d}$ finitarily isomorphic to an i.i.d. process?