Preservation of normality by non-oblivious group selection

TL;DR

This paper proves that non-oblivious group selection methods preserve the normality of sequences, using two different mathematical approaches involving incompressibility and dynamical systems.

Contribution

It introduces two novel proofs demonstrating that non-oblivious group selection preserves sequence normality, expanding understanding of sequence transformations.

Findings

Non-oblivious group selection preserves normality.

Two distinct proofs provided: incompressibility and dynamical systems.

Enhances theoretical understanding of sequence normality preservation.

Abstract

We give two different proofs of the fact that non-oblivious selection via regular group sets preserves normality. Non-oblivious here means that whether or not a symbol is selected can depend on the symbol itself. One proof relies on the incompressibility of normal sequences, the other on the use of augmented dynamical systems.

Equations

$$x\upharpoonright \upharpoonright LA=x\upharpoonright L$$

$$\underset{n\to \infty }{\lim }\frac{|x[\mathrm{1..}n]{|}_w}{n}=(\#A{)}^{-|w|}.$$

$$q_0\stackrel{a_1}{\to }{q}_1\stackrel{a_2}{\to }{q}_2\to \cdots \to q_{n-1}\stackrel{a_n}{\to }{q}_n$$

$$\underset{n\to \infty }{\lim }\frac{\#\{0\le i\le n-1:T^{i}x\in C_w\}}{n}=\mu (C_w).$$

$$\underset{n\to \infty }{\mathrm{limsup}}\frac{\#\{0\le i\le n-1:T^{i}x\in C_w\}}{n}\le c\mu (C_w).$$

$$q_0\stackrel{a_1|v_1}{\to }{q}_1\stackrel{a_2|v_2}{\to }{q}_2\to \cdots \to q_{n-1}\stackrel{a_n|v_n}{\to }{q}_n.$$

$$\underset{n\to \infty }{\mathrm{liminf}}\frac{|v_1{v}_2\cdots v_n|\log \#B}{n\log \#A}<1.$$

$$\underset{n\to \infty }{\mathrm{liminf}}\frac{|c(x[\mathrm{1..}nk])|}{nk}<1.$$

$${\delta }_1(p,a)=\{\begin{array}{ll}(a,\lambda ,\delta (p,a))& \text{if }\delta (p,a)\in F\\ (\lambda ,a,\delta (p,a))& \text{if }\delta (p,a)\notin F.\end{array}$$

$$\underset{n\to \infty }{\mathrm{liminf}}\frac{|c(y[\mathrm{1..}nk])|}{nk}<1.$$

$${\delta }_2((p,w),a)=\{\begin{array}{ll}(\lambda ,\lambda ,(\delta (p,a),wa))& \text{if }\delta (p,a)\in F\text{ and }|wa|<k,\\ (c(wa),\lambda ,(\delta (p,a),\lambda )),& \text{if }\delta (p,a)\in F\text{ and }|wa|=k,\\ (\lambda ,a,(\delta (p,a),w)),& \text{if }\delta (p,a)\notin F.\end{array}$$

$${\delta }_3((p,w_1,w_2,w_3),a)=\{\begin{array}{l}(\lambda ,(\delta (p,a),w_{1}a,w_2,w_3))\\ \text{ if }\delta (p,a)\in F,|w_{1}a|<k\\ (\lambda ,(\delta (p,a),\lambda ,w_{2}c(w_{1}a),w_3))\\ \text{ if }\delta (p,a)\in F,|w_{1}a|=k\\ |w_{2}c(w_{1}a)|<m\\ (0u,(\delta (p,a),\lambda ,v,w_3))\\ \text{ if }\delta (p,a)\in F,|w_{1}a|=k\\ w_{2}c(w_{1}a)=uv\text{ where }|u|=m\\ (\lambda ,(\delta (p,a),w_1,w_2,w_{3}a))\\ \text{ if }\delta (p,a)\notin F,|w_{3}a|<m\\ (1w_{3}a,(\delta (p,a),w_1,w_2,\lambda ))\\ \text{ if }\delta (p,a)\notin F,|w_{3}a|=m\end{array}$$

$$\underset{n\to \infty }{\lim }\frac{\#\{0\le i\le n-1:T^i(x,p)\in C_{w,q}\}}{n}=\tilde{\mu }(C_{w,q}).$$

$$⟨Q_k=Q\times A^k,A,{\delta }_k,I_k,F_k=F\times A^k⟩,$$

$$\{\begin{array}{ll}(ax,({\delta }^{-1}(q,a),w)),& \text{ if }q\notin F,\\ (ax,({\delta }^{-1}(q,a),Aw[\mathrm{1..}k-1])),& \text{ if }q\in F\text{ and }w[k]=a\\ \varnothing ,& \text{ if }q\in F\text{ and }w[k]\ne a\end{array}$$

$$\{\begin{array}{ll}\frac{\mu (E_{q,w})}{\#A}\times \frac{1}{\#Q^{\prime }},& \text{ if }q\notin F,\\ \frac{\mu (E_{q,w})}{\#A}\times \frac{\#A}{\#Q^{\prime }},& \text{ if }q\in F\text{ and }w[k]=a,\\ 0,& \text{ if }q\in F\text{ and }w[k]\ne a.\end{array}$$

$$\tilde{\mu }({\tilde{T}}^{-1}(E_{q,w}\times \{(q,w)\}))=\frac{\mu (E_{q,w})}{\#A}\times \frac{\#A}{\#Q^{\prime }}=\frac{\mu (E_{q,w})}{\#Q^{\prime }}=\tilde{\mu }(E_{q,w}\times \{(q,w)\})$$

$$\underset{m\to \infty }{\mathrm{limsup}}\frac{\#\{0\le i\le m-1:T^{i}y\in C_w\}}{m}\le c\mu (C_w).$$

$$\underset{m\to \infty }{\mathrm{limsup}}\frac{\#\{0\le i\le m-1:T^{i}y\in C_w\}}{m}$$

$$=\underset{m\to \infty }{\mathrm{limsup}}\frac{\#\{0\le i\le i_m-1:{\tilde{T}}^i\tilde{x}\in X\times {\tilde{C}}_w\}}{m}$$

$$=\underset{m\to \infty }{\mathrm{limsup}}\frac{\#\{0\le i\le i_m-1:{\tilde{T}}^i\tilde{x}\in X\times {\tilde{C}}_w\}}{i_m}\cdot {\left(\frac{m}{i_m}\right)}^{-1}.$$

$$\frac{\#\{0\le i\le i_m-1:{\tilde{T}}^i\tilde{x}\in X\times {\tilde{C}}_w\}}{i_m}$$

$$\underset{m\to \infty }{\mathrm{limsup}}\frac{\#\{0\le i\le m-1:T^{i}y\in C_w\}}{m}=\frac{\tilde{\mu }(X\times {\tilde{C}}_w)}{\tilde{\mu }(X\times (Q_k^{\prime }\cap F_k))}.$$

$$\frac{\#{\tilde{C}}_w}{\#(Q_k^{\prime }\cap F_k)}.$$

$$\underset{m\to \infty }{\mathrm{limsup}}\frac{\#\{0\le i\le m-1:T^{i}y\in C_w\}}{m}\le \frac{\#F}{b^k}.$$

Full text

1 Introduction

An infinite sequence $x=x_{1}x_{2}x_{3}x_{4}\dots$ over a finite alphabet $A$ is said to be normal if every finite word appears with the same limiting frequency in $x$ as every other finite word of the same length. (Fuller definitions of terminology used in the introduction will be included in the next section.)

D.D. Wall [20] famously showed that if $x=x_{1}x_{2}x_{3}\dots$ is normal, then $x_{k}x_{k+\ell}x_{k+2\ell}x_{k+3\ell}\dots$ is normal for any $k,\ell\in\mathbb{N}$. In other words, selecting along an arithmetic progression preserves normality. Furstenberg [8], as part of his seminal paper on the theory of disjoint systems, gave another proof of this fact. Kamae [10] and Weiss [21], using the theory of disjoint systems as well, were able to characterize those sequences of positive integers $i_{1}<i_{2}<i_{3}<\dots$ such that selection along these sequences also preserves normality: in particular, these are the deterministic sequences or, equivalently, sequences of Kamae entropy zero.

More generally, many mathematicians have studied (prefix) selection rules. Let $A^{*}$ be the set of finite words over $A$ and let $L\subset A^{*}$. The sequence obtained by oblivious selection of $x$ by $L$ is $x\upharpoonright L=x_{i_{1}}x_{i_{2}}x_{i_{3}}\cdots$, where $i_{1},i_{2},i_{3},\ldots$ is the enumeration in increasing order of all the integers $i$ such that the prefix $x_{1}x_{2}\cdots x_{i-1}$ belongs to $L$. This selection rule is called oblivious because the symbol $x_{i}$ is not included in the considered prefix. If $L=A^{*}1$ is the set of words ending with a $1$, the sequence $x\upharpoonright L$ is made of all symbols of $x$ occurring after a $1$ in the same order as they occur in $x$.

The examples above by Wall, Kamae, and Weiss are all examples of oblivious selection rules, where $L$ consists of all words of certain fixed lengths. However, far more intricate selection rules are possible. The following theorem of Agafonov [1] states that normality is preserved by oblivious selection of a regular language.

Theorem 1** (Agafonov).**

If the sequence $x\in A^{\mathbb{N}}$ is normal and $L\subset A^{*}$ is regular, then $x\upharpoonright L$ is also normal.

A language $L\subset A^{*}$ is regular if it is accepted by a deterministic finite automaton. We will speak more on this later.

Kamae and Weiss [11] extended Theorem 1 slightly. Let $L$ be a set of words and let $\sim_{L}$ be an equivalence relation given by $u\sim_{L}v$ if $\{w:uw\in L\}=\{w:vw\in L\}$. If $L/\sim_{L}$ is finite, then selection along $L$ preserves normality. In contrast, Merkle and Reimann [12] showed that selection by deterministic one-counter languages or by linear languages need not preserve normality. We also mention that suffix selection, where the selection of a given digit is based of the tail of the sequence after that digit, has also been considered [3].

We can also define the sequence obtained by non-oblivious selection of $x$ by $L$. This is $x\upharpoonright\mkern-4.0mu\upharpoonright L=x_{i_{1}}x_{i_{2}}x_{i_{3}}\cdots$, where $i_{1},i_{2},i_{3},\ldots$ is the enumeration in increasing order of all the integers $i$ such that the prefix $x_{1}x_{2}x_{3}\cdots x_{i}$ including $x_{i}$ belongs to $L$.

Non-oblivious selection is more powerful than oblivious selection, as it can simulate the latter due to the following formula:

[TABLE]

for any sequence $x$ and any set $L$ of words. Let us recall that $LA$ is the set of words of the form $wa$ for $w\in L$ and $a\in A$. On the other hand, there are weaknesses in non-oblivious selection as well. If we take $L=A^{*}1$ again, then $x\upharpoonright\mkern-4.0mu\upharpoonright L$ will consist of nothing but $1$’s, which will not be normal.

Accordingly, oblivious selection has been studied more than non-oblivious selection. The second author [19] has a few (very specific) examples of non-oblivious selections that preserve normality. In this paper, we present a more general theorem:

Theorem 2**.**

If the sequence $x\in A^{\mathbb{N}}$ is normal and $L\subset A^{*}$ is a regular group set, then $x\upharpoonright\mkern-4.0mu\upharpoonright L$ is also normal.

Regular here has the same meaning as in Agafonov’s theorem. Saying that $L$ is a group set implies that in the associated deterministic finite automaton, any input will permute the states.

We note that if there exists a regular group set $K$ such that the symmetric difference $L\triangle K$ is finite, then the non-oblivious selection by $L$ also preserves normality. More generally, if $L$ is accepted by an automaton such that each recurrent strongly connected component (those components, which, once entered, cannot be left) is a group automaton, then non-oblivious selection by $L$ still preserves normality.

We will prove this result using two distinct methods reflecting the different styles of the two authors of this paper.

The first method, favored by the first author, makes use of the fact that normality can be defined in terms of incompressibility by deterministic finite automata [3]. It follows along the lines of the proof of Agafonov’s theorem presented in [2]. One key ingredient of this proof is the statement that the function which maps each sequence $x$ to the pair $(x\upharpoonright L,x\upharpoonright(A^{*}\setminus L))$ is one-to-one. The same statement for the non-oblivious selection does not hold, even when $L$ is a group set as is shown by the following example. Consider the set $L$ of words accepted by the automaton pictured in Figure 1. Let $x$ and $x^{\prime}$ be the sequences $01^{\mathbb{N}}$ and $101^{\mathbb{N}}$. It is easily computed that $x\upharpoonright\mkern-4.0mu\upharpoonright L=x^{\prime}\upharpoonright\mkern-4.0mu\upharpoonright L=01^{\mathbb{N}}$ and $x\upharpoonright\mkern-4.0mu\upharpoonright(A^{*}\setminus L)=x^{\prime}\upharpoonright\mkern-4.0mu\upharpoonright(A^{*}\setminus L)=1^{\mathbb{N}}$.

The second method, favored by the second author, makes use of the idea of augmented systems, dynamical systems which have been extended to simultaneously act over a deterministic finite automaton [9, 18]. In this framework, the distinction between oblivious and non-oblivious selection is much smaller. We make use of a technique from [19], where we use an automaton that also records a finite number of selected symbols: this reduces the problem of counting frequencies of words in $x\upharpoonright\mkern-4.0mu\upharpoonright L$ to the problem of calculating visiting frequencies of certain states in the automaton.

In Section 2, we will give definitions, notation, and the necessary results from previous papers. In Section 3, we provide a proof of Theorem 2, following the ideas of incompressibility. In Section 4, we provide a proof of Theorem 2, following the ideas of augmented systems.

2 Preliminaries

2.1 Sequences, words, and normality

We write ${\mathbb{N}}=\{1,2,3,\dots\}$ for the set of all natural numbers. An alphabet $A$ is a finite set with at least two symbols. We respectively write $A^{*}$ and $A^{{\mathbb{N}}}$ for the set of all finite sequences (also known as words) and the set of all infinite sequences of elements of $A$ (which we will simply refer to as sequences). We also write $A^{k}$ stands for the set of all words of length $k$. The length of a finite word $w$ is denoted by $|w|$. The empty word is denoted by $\lambda$. The positions in finite and infinite sequences are numbered starting from $1$. For a word $w$ and positions $1\leq i\leq j\leq|w|$, we let $w[i]$ and $w[i..j]$ denote the symbol $a_{i}$ at position $i$ and the word $a_{i}a_{i+1}\cdots a_{j}$ from position $i$ to position $j$. A set $C$ of words is called prefix-free if for any $u,v\in C$ with $u$ being a prefix of $v$ ($u=v[1..|u|]$), we have $u=v$. We write $\log$ for the base $2$ logarithm. For any finite set $S$ we denote its cardinality with $\#S$.

Let $x=a_{1}a_{2}a_{3}\cdots$ be a sequence over the alphabet $A$. Let $L\subseteq A^{*}$ be a set of words over $A$. As described in the introduction, the sequence obtained by oblivious selection of $x$ by $L$ is $x\upharpoonright L=a_{i_{1}}a_{i_{2}}a_{i_{3}}\cdots$, where $i_{1},i_{2},i_{3},\ldots$ is the enumeration in increasing order of all the integers $i$ such that the prefix $a_{1}a_{2}\cdots a_{i-1}$ belongs to $L$. The sequence obtained by non-oblivious selection of $x$ by $L$ is $x\upharpoonright\mkern-4.0mu\upharpoonright L=a_{i_{1}}a_{i_{2}}a_{i_{3}}\cdots$ where $i_{1},i_{2},i_{3},\ldots$ is the enumeration in increasing order of all the integers $i$ such that the prefix $a_{1}a_{2}a_{3}\cdots a_{i}$ including $a_{i}$ belongs to $L$.

We recall here the notion of normality. We start with the notation for the number of occurrences of a given word $u$ within another word $w$. For $w$ and $u$ two words, the number $|w|_{u}$ of occurrences of $u$ in $w$ is given by $|w|_{u}=\#\{i:w[i..i+|u|-1]=u\}$. We say that $x\in A^{\mathbb{N}}$ is normal if for each word $w\in A^{*}$, we have

[TABLE]

This differs from Borel’s original definition [5] of normality, but is equivalent (see [2, Sect. 7.3]).

2.2 Deterministic finite automata

A deterministic finite automaton is a tuple $\mathcal{T}=\langle Q,A,\delta,I,F\rangle$, where $Q$ is a finite set of states, $A$ is the input alphabet, $\delta:Q\times A\to Q$ is the transition function, and $I\subseteq Q$ and $F\subseteq Q$ are the sets of initial and final states, respectively. We focus on automata that operate in real-time, that is, they process exactly one input alphabet symbol per transition. Moreover, we will assume that there is a single initial state, that is, $I$ is a singleton set.

The relation $\delta(p,a)=q$ is written p\mathchoice{\xrightarrow{a}}{\xrightarrow{\smash{\lower 1.0pt\hbox{\scriptstyle a}}}}{\text{Error}}{\text{Error}}q and we further denote the sequence of consecutive transitions

[TABLE]

by q_{0}\mathchoice{\xrightarrow{a_{1}a_{2}\dots a_{n}}}{\xrightarrow{\smash{\lower 1.0pt\hbox{\scriptstyle a_{1}a_{2}\dots a_{n}}}}}{\text{Error}}{\text{Error}}q_{n}. A word $w=a_{1}a_{2}\dots a_{n}$ is said to be accepted by an automaton if q_{0}\mathchoice{\xrightarrow{w}}{\xrightarrow{\smash{\lower 1.0pt\hbox{\scriptstyle w}}}}{\text{Error}}{\text{Error}}q_{n}, $q_{0}$ is the initial state (that is, $I=\{q_{0}\}$), and $q_{n}$ is final (that is, in $F$). A language $L\subset A^{*}$ is said to be regular (as seen in Theorem 1) if there exists a deterministic finite automaton $\mathcal{T}$ such that $w\in L$ if and only if $w$ is accepted by $\mathcal{T}$.

We now introduce a classical class of regular sets called group sets (as seen in Theorem 2). A group automaton is a deterministic automaton such that each symbol induces a permutation of the states. By inducing a permutation, we mean that, for each symbol $a$, the function which maps each state $p$ to the state $q$ such that p\mathchoice{\xrightarrow{a}}{\xrightarrow{\smash{\lower 1.0pt\hbox{\scriptstyle a}}}}{\text{Error}}{\text{Error}}q is a permutation of the state set. Put another way, if p\mathchoice{\xrightarrow{a}}{\xrightarrow{\smash{\lower 1.0pt\hbox{\scriptstyle a}}}}{\text{Error}}{\text{Error}}q and p^{\prime}\mathchoice{\xrightarrow{a}}{\xrightarrow{\smash{\lower 1.0pt\hbox{\scriptstyle a}}}}{\text{Error}}{\text{Error}}q are two transitions of the automaton, then $p=p^{\prime}$. A regular set $L\subseteq A^{*}$ is called a group set if $L$ is accepted by a group automaton. It is well known that a regular set is a group set if and only if its syntactic monoid is a group [15, Sect. 7.5].

2.3 Dynamical systems

We will consider dynamical systems to consist of a tuple $\mathcal{X}=\langle X,\mathcal{F},T,\mu\rangle$, where $X$ is a space, $\mathcal{F}$ is a $\sigma$-algebra on this space, $T:X\to X$ is a transformation (always assumed to be continuous with respect to $\mathcal{F}$), and $\mu$ is a measure on $\mathcal{F}$. We say that $T$ preserves the measure $\mu$, or, equivalently, that $\mu$ is $T$-invariant, if $\mu(T^{-1}A)=\mu(A)$ for all $A\in\mathcal{F}$. We say that a system is ergodic if $\mu(T^{-1}A\triangle A)=0$ for some $A\in\mathcal{F}$ implies that $\mu(A)=0$ or $\mu(X\setminus A)=0$. Ergodicity may be considered an indecomposability criterion for dynamical systems: a system is ergodic if it cannot be split into two large $T$-invariant pieces.

Of particular relevance to this paper is the symbolic shift system. Let $X=A^{\mathbb{N}}$ be the space of all infinite words on the alphabet $A$. Let $T$ denote the forward shift on $X$, so that for a sequence $x\in X$, $(Tx)_{i}=x_{i+1}$. Given a word $w\in A^{*}$, let $C_{w}$ denote the cylinder set corresponding to $w$, so that $C_{w}$ consists of all $x\in X$ such that $x[1..|w|]=w$. The cylinder sets form a semi-algebra that generates the canonical $\sigma$-algebra on $X$ and so if we let $\mu$ be a measure on cylinder sets $C_{w}$ given by $\mu(C_{w})=(\#A)^{-|w|}$, then this extends to a measure on this $\sigma$-algebra. We note that if $A=\{0,1,\dots,b-1\}$ and we use the standard bijection111Actually not quite a bijection, due to the phenomenon of $0.0\overline{9}=0.1$, but this will not be relevant and can be ignored. from $X$ to $[0,1)$ associating an infinite word with a $b$-ary expansion, then $\mu$ is just the Lebesgue measure.

We may reinterpret the definition of normality in a slightly more ergodic manner. Since $x[i..i+|w|-1]=w$ if and only if $T^{i-1}x\in C_{w}$, we have that a word $x\in X$ is normal if for each finite word $w\in A^{*}$, we have

[TABLE]

Note that $T$ is ergodic and invariant with respect to $\mu$, so by the pointwise ergodic theorem, $\mu$-almost all $x\in X$ are normal.

By the Pyatetskii-Shapiro normality criterion [13], we can weaken the above to say that $x$ is normal if there exists a fixed constant $c>0$ such that for every $w\in A^{*}$, we have

[TABLE]

3 The first method of proof

We now introduce automata with output, also known as transducers, which are used to compress sequences and to select symbols from a sequence. In this paper we only consider input-deterministic transducers (also known as sequential) computing functions from sequences to sequences. Such a machine is a deterministic automaton in which each transition is equipped with an additional output word. The output of a (infinite) run is the concatenation of the outputs of the transitions used by the run. More formally a transducer is a tuple $\mathcal{T}=\langle Q,A,B,\delta,I,F\rangle$, where $Q$ is a finite set of states, $A$ and $B$ are the input and output alphabets, respectively, $\delta:Q\times A\to B^{*}\times Q$ is the transition function, and $I\subseteq Q$ and $F\subseteq Q$ are the sets of initial and final states, respectively. The set $I$ is again a singleton set.

The relation $\delta(p,a)=(w,q)$ is written p\mathchoice{\xrightarrow{a|w}}{\xrightarrow{\smash{\lower 1.0pt\hbox{\scriptstyle a|w}}}}{\text{Error}}{\text{Error}}q and the tuple $\langle p,a,w,q\rangle$ is then called a transition of the transducer. A finite (respectively, infinite) run is a finite (respectively, infinite) sequence of consecutive transitions,

[TABLE]

Its input and output labels are respectively $a_{1}\cdots a_{n}$ and $v_{1}\cdots v_{n}$. A finite run is written q_{0}\mathchoice{\xrightarrow{a_{1}\cdots a_{n}|v_{1}\cdots v_{n}}}{\xrightarrow{\smash{\lower 1.0pt\hbox{\scriptstyle a_{1}\cdots a_{n}|v_{1}\cdots v_{n}}}}}{\text{Error}}{\text{Error}}q_{n}. An infinite run is final if the state $q_{n}$ is final for infinitely many integers $n$. In that case, the infinite run is written q_{0}\mathchoice{\xrightarrow{a_{1}a_{2}a_{3}\cdots|v_{1}v_{2}v_{3}\cdots}}{\xrightarrow{\smash{\lower 1.0pt\hbox{\scriptstyle a_{1}a_{2}a_{3}\cdots|v_{1}v_{2}v_{3}\cdots}}}}{\text{Error}}{\text{Error}}\infty. An infinite run is accepting if it is final and furthermore its first state $q_{0}$ is the initial one. This is the classical Büchi acceptance condition [14]. Since transducers are supposed to be input-deterministic, there is at most one accepting run q_{0}\mathchoice{\xrightarrow{x|y}}{\xrightarrow{\smash{\lower 1.0pt\hbox{\scriptstyle x|y}}}}{\text{Error}}{\text{Error}}\infty having a given sequence $x$ for input label and we write $y=\mathcal{T}(x)$.

A transducer is called one-to-one if the function which maps $x$ to $y$ is one-to-one. We always assume that all transducers are trim: each state occurs in at least one accepting run.

A sequence $x=a_{1}a_{2}a_{3}\cdots$ is compressible by a transducer $\mathcal{T}$ if it has an accepting run q_{0}\mathchoice{\xrightarrow{a_{1}|v_{1}}}{\xrightarrow{\smash{\lower 1.0pt\hbox{\scriptstyle a_{1}|v_{1}}}}}{\text{Error}}{\text{Error}}q_{1}\mathchoice{\xrightarrow{a_{2}|v_{2}}}{\xrightarrow{\smash{\lower 1.0pt\hbox{\scriptstyle a_{2}|v_{2}}}}}{\text{Error}}{\text{Error}}q_{2}\mathchoice{\xrightarrow{a_{3}|v_{3}}}{\xrightarrow{\smash{\lower 1.0pt\hbox{\scriptstyle a_{3}|v_{3}}}}}{\text{Error}}{\text{Error}}q_{3}\cdots satisfying

[TABLE]

Recall that each of the $v_{i}$’s belongs to $B^{*}$, not necessarily $B$, so could be empty or have length greater than $1$.

The connection between compressible sequences and normality is given by the following:

Theorem 3**.**

A sequence is normal if and only if it not compressible by a one-to-one deterministic transducer.

The above result follows from the results in [7, 16]. A direct proof appears in [4]. Extensions of this characterization for non-deterministic and extra memory transducers are in [3, 6].

Let $c:A^{k}\to A^{*}$ be a function mapping each word of length $k$ to some word. This function can be extended to a function from $(A^{k})^{*}$ to $A^{*}$ by setting $c(w_{1}\cdots w_{n})=c(w_{1})\cdots c(w_{n})$ with $w_{i}\in A^{k}$ for $1\leq i\leq n$. When a sequence $x$ is not normal, it can be compressed using a Huffman coding. This is implicit in the following lemma. The proof of the next lemma is the first part of the proof of Lemma 7.5.1 in [2].

Lemma 4**.**

If the sequence $x$ is not normal, there is a length $k$ and a one-to-one function $c:A^{k}\to C$ where $C$ is a prefix-free set such that

[TABLE]

The next lemma states that if the input $x$ is normal, each state which is visited infinitely often in the run over $x$ in some deterministic automaton is visited more than a linear number of times. This lemma is actually Lemma 7.10.3 in [2].

Lemma 5**.**

Let $x=a_{0}a_{1}a_{2}\cdots$ be a normal sequence and let q_{0}\mathchoice{\xrightarrow{a_{0}}}{\xrightarrow{\smash{\lower 1.0pt\hbox{\scriptstyle a_{0}}}}}{\text{Error}}{\text{Error}}q_{1}\mathchoice{\xrightarrow{a_{1}}}{\xrightarrow{\smash{\lower 1.0pt\hbox{\scriptstyle a_{1}}}}}{\text{Error}}{\text{Error}}q_{2}\mathchoice{\xrightarrow{a_{2}}}{\xrightarrow{\smash{\lower 1.0pt\hbox{\scriptstyle a_{2}}}}}{\text{Error}}{\text{Error}}\cdots be a run in a deterministic automaton. If the state $q$ is visited infinitely often in this run, then $\liminf_{n\to\infty}\#\{i\leq n:q_{i}=q\}/n>0$.

A transducer can have two output tapes. In this case, the transducer would be a tuple $\mathcal{T}=\langle Q,A,B,\delta,I,F\rangle$, where now the transition function is $\delta:Q\times A\to B^{*}\times B^{*}\times Q$. The relation $\delta(p,a)=(w_{1},w_{2},q)$ is written p\mathchoice{\xrightarrow{a|w_{1},w_{2}}}{\xrightarrow{\smash{\lower 1.0pt\hbox{\scriptstyle a|w_{1},w_{2}}}}}{\text{Error}}{\text{Error}}q.

A deterministic automaton $\mathcal{A}$ accepting a set $L$ can can be turned into a two-output transducer $\mathcal{T}$ that outputs $x\upharpoonright\mkern-4.0mu\upharpoonright L$ and $x\upharpoonright\mkern-4.0mu\upharpoonright(A^{*}\setminus L)$ on its first and second output tapes respectively. Each transition p\mathchoice{\xrightarrow{a}}{\xrightarrow{\smash{\lower 1.0pt\hbox{\scriptstyle a}}}}{\text{Error}}{\text{Error}}q of $\mathcal{A}$ is replaced by either the transition p\mathchoice{\xrightarrow{a|a,\lambda}}{\xrightarrow{\smash{\lower 1.0pt\hbox{\scriptstyle a|a,\lambda}}}}{\text{Error}}{\text{Error}}q if the state $q$ is final or by the transition p\mathchoice{\xrightarrow{a|\lambda,a}}{\xrightarrow{\smash{\lower 1.0pt\hbox{\scriptstyle a|\lambda,a}}}}{\text{Error}}{\text{Error}}q if $q$ is not final. If $q$ is final, any finite run in $\mathcal{A}$ from the initial state to $q$ is accepting and therefore the label $a$ of its last transition must be output to the first output tape because it is selected in $x\upharpoonright\mkern-4.0mu\upharpoonright L$. The following lemma states the key property of this transducer $\mathcal{T}$ when $\mathcal{A}$ is a group automaton.

Lemma 6**.**

Let $\mathcal{A}$ be a group automaton and let $\mathcal{T}$ the transducer obtained from $\mathcal{A}$ as above. The function which maps each finite run p\mathchoice{\xrightarrow{u|v,w}}{\xrightarrow{\smash{\lower 1.0pt\hbox{\scriptstyle u|v,w}}}}{\text{Error}}{\text{Error}}q of $\mathcal{T}$ to the triple $\langle q,v,w\rangle$ is one-to-one.

Proof.

Note first that $|u|=|v|+|w|$. The proof is done by induction on the sum $|v|+|w|$. We will, throughout the proof, denote the run p\mathchoice{\xrightarrow{u|v,w}}{\xrightarrow{\smash{\lower 1.0pt\hbox{\scriptstyle u|v,w}}}}{\text{Error}}{\text{Error}}q by $\rho$.

If $v$ and $w$ are empty, $u$ is also empty and so $\rho$ must be the empty run from $p=q$ to $p$. Suppose now that $|v|+|w|>0$. Suppose that $q$ is final in the automaton $\mathcal{A}$. All transitions ending in $q$ have the form r\mathchoice{\xrightarrow{a|a,\lambda}}{\xrightarrow{\smash{\lower 1.0pt\hbox{\scriptstyle a|a,\lambda}}}}{\text{Error}}{\text{Error}}q for some state $r$ and some symbol $a$. It follows that $v$ cannot be empty. Let $a$ be the last symbol of $v$ and let $v^{\prime}$ be such that $v=v^{\prime}a$. Since $\mathcal{A}$ is a group automaton, there is exactly one transition r\mathchoice{\xrightarrow{a}}{\xrightarrow{\smash{\lower 1.0pt\hbox{\scriptstyle a}}}}{\text{Error}}{\text{Error}}q ending in $q$ and having $a$ for label. This implies that the last transition of $\rho$ is r\mathchoice{\xrightarrow{a|a,\lambda}}{\xrightarrow{\smash{\lower 1.0pt\hbox{\scriptstyle a|a,\lambda}}}}{\text{Error}}{\text{Error}}q. Applying the induction hypothesis to the triple $\langle r,v^{\prime},w\rangle$ completes the proof in this case. The case where $q$ is not final in $\mathcal{A}$ works similarly. ∎

Proof of Theorem 2.

Let $x$ be a normal word. Let $L\subset A^{*}$ be a regular group set. We suppose that $x\upharpoonright\mkern-4.0mu\upharpoonright L$ is not normal and will show that $x$ can be compressed by a one-to-one deterministic transducer, contradicting its normality.

First note that $x\upharpoonright\mkern-4.0mu\upharpoonright(A^{*}\setminus L)$ cannot be a finite word, as otherwise the normality of $x\upharpoonright\mkern-4.0mu\upharpoonright L$ is trivial.

Let $\mathcal{A}$ be a group automaton accepting $L$ whose state set and transition function are $Q$ and $\delta$ respectively. Let $\mathcal{T}_{1}$ be the two-output transducer obtained from $\mathcal{A}$ as above. The state set of $\mathcal{T}_{1}$ is the same as the one of $\mathcal{A}$ and the transition function $\delta_{1}$ of $\mathcal{T}_{1}$ is defined as follows.

[TABLE]

Since this transducer is assumed to be trim and since $x$ is assumed to be normal, it must reach final states infinitely often, showing that $x\upharpoonright\mkern-4.0mu\upharpoonright L$ is infinite. This is a consequence of [16, Satz 2.5].

Since it is supposed that $y=x\upharpoonright\mkern-4.0mu\upharpoonright L$ is not normal, there is, by Lemma 4, an integer $k$ and a one-to-one function $c$ from $A^{k}$ into a prefix-free set $C$ such that

[TABLE]

The transducer $\mathcal{T}_{1}$ can be combined with the function $c$ to get a new transducer $\mathcal{T}_{2}$ which writes $c(y[1..nk])$ instead of $y[1..nk]$ on its first output tape. This latter transducer has a buffer $\mathcal{B}=\mathcal{B}(k)$ of size $k$ in which each symbol of $y$ is put. Whenever this buffer becomes full containing a word $w$, the transducer $\mathcal{T}_{2}$ writes $c(w)$ to its first output tape. $\mathcal{T}_{2}$ also behaves identically to $\mathcal{T}_{1}$ on its second output tape. To be more precise, the state set of $\mathcal{T}_{2}$ is $Q\times\mathcal{B}(k)$ where $\mathcal{B}(k)=\bigcup_{i=0}^{k-1}B^{i}$. The transition function $\delta_{2}$ of $\mathcal{T}_{2}$ is defined as follows.

[TABLE]

We claim that $\mathcal{T}_{2}$ compresses its input $x$. By this, we mean that if $u_{n}$ and $v_{n}$ are the outputs of $\mathcal{T}_{2}$ on its two tapes after $n$ transitions (consuming $n$ input symbols), then $\liminf_{n\to\infty}{(|u_{n}|+|v_{n}|)/n}<1$. The result is clear because the coding function $c$ compresses the word $y$ and because, by Lemma 5, along the run of the automaton $\mathcal{A}$, final states are visited at linearly many times.

We now construct a new transducer $\mathcal{T}_{3}$ which merges the contents of the two output tapes of $\mathcal{T}_{2}$ into a single output. Let $m$ be a integer to be fixed later. The transducer $\mathcal{T}_{3}$ has one output tape and two buffers $\mathcal{B}_{1}=\mathcal{B}(m)$ and $\mathcal{B}_{2}=\mathcal{B}(m)$ of size $m$, constructed in a similar way to $\mathcal{T}_{2}$ above. Whenever $\mathcal{T}_{2}$ writes a symbol to its first (respectively, second) output tape, this symbol is added to the buffer $\mathcal{B}_{1}$ (respectively, $\mathcal{B}_{2}$). Whenever one buffer $\mathcal{B}_{1}$ or $\mathcal{B}_{2}$ becomes full, its content is written to the output tape of $\mathcal{T}_{3}$ with an extra bit in front of it to indicate whether the content comes from either $\mathcal{B}_{1}$ or $\mathcal{B}_{2}$. This extra bit is [math] if the content comes from $\mathcal{B}_{1}$ and $1$ otherwise. To be more precise, the state set of $\mathcal{T}_{3}$ is $Q\times\mathcal{B}(k)\times\mathcal{B}(m)\times\mathcal{B}(m)$ and its transition function $\delta_{3}$ is defined as follows.

[TABLE]

We assume that $m$ is sufficiently large so that $|c(w)|\leq m$ for any $w\in A^{k}$: this guarantees that in the third case above, we have $m\leq w_{2}c(w_{1}a)<2m$ so $v$ is in $\mathcal{B}(m)$ as desired. Moreover, we claim that, for $m$ great enough, the transducer $\mathcal{T}_{3}$ also compresses its input $x$. Note that the output of $\mathcal{T}_{3}$ is longer than the sum of the two outputs of $\mathcal{T}_{2}$ because each block of $m$ symbols is preceded by an extra bit [math] or $1$. However, for $m$ great enough, this loss is offset by the compression of $\mathcal{T}_{2}$, and indeed $\mathcal{T}_{3}$ compresses its input.

Note that none of the transducers $\mathcal{T}_{1}$, $\mathcal{T}_{2}$, and $\mathcal{T}_{3}$ is one-to-one because the function which maps $x$ to the pair $(x\upharpoonright\mkern-4.0mu\upharpoonright L,x\upharpoonright\mkern-4.0mu\upharpoonright(A^{*}\setminus L))$ might not be one-to-one. For that reason, we construct a last transducer $\mathcal{T}_{4}$ obtained by changing $\mathcal{T}_{3}$ to make it one-to-one. The transducer $\mathcal{T}_{4}$ works as $\mathcal{T}_{3}$ but whenever $\mathcal{T}_{3}$ writes to its output tape a block of length $m$ coming from its buffer $\mathcal{B}_{1}$ with its extra bit [math], the transducer $\mathcal{T}_{4}$ also writes some extra information that we now describe. This extra information is made of two data. The first one is the current state $\delta(p,a)$ of the automaton $\mathcal{A}$. This other one is the length of the buffer $\mathcal{B}_{2}$. Both data are written in binary and require $\lceil\log\#Q\rceil$ and $\lceil\log m\rceil$ bits respectively. We do not give explicitly the transition function of $\mathcal{T}_{4}$ as it is almost the same as the one of $\mathcal{T}_{3}$.

Since the additional information written by $\mathcal{T}_{4}$ is $o(m)$, $\mathcal{T}_{4}$ still compresses its input for $m$ large enough. We also claim that the transducer $\mathcal{T}_{4}$ is one-to-one. To do this, we show that, from the output of $\mathcal{T}_{4}$, it is possible to recover the input.

Consider any block in the output which comes from $\mathcal{B}_{1}$ (indicated by the first binary bit in front of it). If we take this block and all preceding blocks that arise from $\mathcal{B}_{1}$, apply $c^{-1}$ to them, and concatenate them in order, we obtain a prefix $v$ of $x\upharpoonright\mkern-4.0mu\upharpoonright L$. In a similar way, if we take all preceding blocks that arise from $\mathcal{B}_{2}$ and the numbers of symbols of the next $\mathcal{B}_{2}$ block indicated by the second data, we obtain a prefix $w$ of $x\upharpoonright\mkern-4.0mu\upharpoonright(A^{*}\setminus L)$. (Recall our assumption that $x\upharpoonright\mkern-4.0mu\upharpoonright(A^{*}\setminus L)$ is infinite, so this next block must always exist.) Finally, the first data gives us a state $q$ for our original automaton $\mathcal{A}$. From the way that $\mathcal{T}_{3}$ is constructed, this specific triple $\langle q,v,w\rangle$, represents the outputs and state reached by a run q_{0}\mathchoice{\xrightarrow{u|v,w}}{\xrightarrow{\smash{\lower 1.0pt\hbox{\scriptstyle u|v,w}}}}{\text{Error}}{\text{Error}}q. By Lemma 6, the triple $\langle q,v,w\rangle$ uniquely identifies this $u$ and means it must necessarily be a prefix of $x$. Since there are infinitely many blocks that come from $\mathcal{B}_{1}$, this gives us infinitely many prefixes of $x$ and so we know the entirety of $x$.

∎

4 The second method of proof

We now consider what happens when we augment a dynamical system to simultaneously run over a finite state automaton. Let us consider, as before, a dynamical system $\langle X,\mathcal{F},T,\mu\rangle$ and an automaton $\langle Q,A,\delta,I,F\rangle$. We will say this automaton is transitive if given any $q_{1},q_{2}\in Q$ there exists a word $w\in A^{*}$ such that q_{1}\mathchoice{\xrightarrow{w}}{\xrightarrow{\smash{\lower 1.0pt\hbox{\scriptstyle w}}}}{\text{Error}}{\text{Error}}q_{2}.

We now consider the following augmented dynamical system $\langle\widetilde{X},\widetilde{\mathcal{F}},\widetilde{T},\tilde{\mu}\rangle$:

$\widetilde{X}:=X\times Q$, 2. 2.

$\widetilde{T}:\widetilde{X}\to\widetilde{X}$ given by $\widetilde{T}(x,q)=(Tx,\delta(q,x_{1}))$, where $x=x_{1}x_{2}x_{3}\dots$, 3. 3.

Cylinder sets $C_{w,q}=C_{w}\times\{q\}$ for $w\in A^{*}$, $q\in Q$ (noting again that these cylinder sets generate the $\sigma$-algebra $\widetilde{\mathcal{F}}$ on $\widetilde{X}$), 4. 4.

$\tilde{\mu}(C_{w,q})=(\#A)^{-|w|}/(\#Q)$.

We extend our definition of normality on augmented systems: $(x,p)\in\widetilde{X}$ is said to be normal if for every $w\in A^{*}$ and $q\in Q$, we have

[TABLE]

We have then the following result:

Theorem 7**.**

Suppose that the automaton $\langle Q,A,\delta,F\rangle$ is a transitive automaton. If $\widetilde{T}$ preserves the measure $\tilde{\mu}$, then $\widetilde{T}$ is ergodic. Moreover, for any $x\in X$ that is normal, the point $(x,q)$ is normal w.r.t $\tilde{\mu}$ for any $q\in Q$.

This is a simplified verison of Theorem 3.1 in [18] (see Remark 3.2 in that paper for the discussion of the necessary conditions needed on the dynamic system and note that they are all trivial in our case). See also [17] for a simpler proof.

We again want to consider adding a buffer to an automaton; however, unlike in the previous section, we wish to consider a “rolling” buffer, which will continuously record the previous few inputs that caused us to reach a final state without resetting itself to a shorter word.

For a given $k\in\mathbb{N}$ and automaton $\langle Q,A,\delta,I,F\rangle$ as above, consider the automaton

[TABLE]

where $\delta_{k}$ satisfies the following rules:

• •

If $\delta(q,a)$ is not final, then $\delta_{k}((q,w),a)=(\delta(q,a),w)$.

• •

If $\delta(q,a)$ is final, then $\delta_{k}((q,w),a)=(\delta(q,a),w[2..k-1]a)$,

and $I_{k}=\{(q_{0},w_{0})\}$, where $w_{0}$ is any element of $A^{k}$. The choice of which $w_{0}$ to use will not be relevant for any subsequent proofs. We will refer this new automaton as a $k$-digit buffer over the original automaton.

Lemma 8**.**

Let $x\in X$ be normal and $q\in Q_{k}$, and let $\langle Q_{k},A,\delta_{k},I_{k},F_{k}\rangle$ be a $k$-digit buffer over a transitive group automaton. Then there is a subset $Q^{\prime}\subseteq Q_{k}$ such that $F_{k}\cap Q^{\prime}$ is non-empty, $\langle Q^{\prime},A,\delta_{k},I_{k}\cap Q^{\prime},F_{k}\cap Q^{\prime}\rangle$ is a transitive automaton , and $\widetilde{T}^{i}(x,q)$ will eventually always be in $Q^{\prime}$ in its second coordinate. Moreover, every word in $A^{k}$ will appear in the second coordinate of some element of $Q^{\prime}$.

We are being somewhat imprecise about the initial states in the new automaton $\langle Q^{\prime},A,\delta_{k},I_{k}\cap Q^{\prime},F_{k}\cap Q^{\prime}\rangle$. It is possible that $I_{k}\cap Q^{\prime}=\emptyset$. In this case we would replace $I_{k}\cap Q^{\prime}$ with the first state in $Q^{\prime}$ that the $\widetilde{T}$-orbit of $(x,q)$ enters—in essence, shifting everything forward.

Proof.

Most of this is proved in Lemma 4.1 of [18] with the exception the non-emptiness of $F_{k}\cap Q^{\prime}$ and the last line.

Consider any $(q_{1},w)\in Q^{\prime}$ and $q_{2}\in Q$. Since $\langle Q,A,\delta,I,F\rangle$ is transitive, there exists another word, $u\in A^{*}$, so that q_{1}\mathchoice{\xrightarrow{u}}{\xrightarrow{\smash{\lower 1.0pt\hbox{\scriptstyle u}}}}{\text{Error}}{\text{Error}}q_{2} in this automaton. Moreover, since $\langle Q_{k},A,\delta_{k},I_{k},F_{k}\rangle$ behaves the same as $\langle Q,A,\delta,I,F\rangle$ in the first coordinate, we must have that (q_{1},w)\mathchoice{\xrightarrow{u}}{\xrightarrow{\smash{\lower 1.0pt\hbox{\scriptstyle u}}}}{\text{Error}}{\text{Error}}(q_{2},w_{2}) for some word $w_{2}\in A^{k}$. Since $q_{2}$ is an arbitrary element of $Q$, we have that every element of $Q$ appears in the first coordinate of $Q^{\prime}$. In particular, $F_{k}\cap Q^{\prime}$ is non-empty.

We will now show that every word in $A^{k}$ will appear in the second coordinate of some element of $Q^{\prime}$. Note that if one starts at a state $(q,w)\in F_{k}\cap Q^{\prime}$, it is always possible to reach the next final state in $F_{k}\cap Q^{\prime}$ via any element in $A$. In particular, $\langle Q,A,\delta,I,F\rangle$ itself is a group automaton and so the action of any input is to permute the states. Thus if one starts at $q$ and keeps repeating the input $a_{1}$, one must eventually arrive at state $q_{1}\in F$. At worst the permutation that $a_{1}$ induces on the states $Q$ has $q$ being the only final state in its cycle. But even in this case, we would just have $q_{1}=q$. Thus, in our buffered automaton, we have that by inputting $a_{1}$ enough times we will move from $(q,w)$ to $(q_{1},w[2..k]a_{1})\in Q^{\prime}$. We may repeat this process by inputting $a_{2}$ over and over until we reach a state $(q_{2},w[3..k]a_{1}a_{2})\in Q^{\prime}$, and so on, until we reach $(q_{k},a_{1}a_{2}\dots a_{k})\in Q^{\prime}$. But since the $a_{i}$’s are all arbitrary, one can force any desired word to appear in the second coordinate of $Q^{\prime}$. ∎

Lemma 9**.**

Suppose we augment $(X,T,\mu)$ with $\langle Q^{\prime},A,\delta_{k},I_{k},F_{k}\rangle$ as defined in the previous lemma. Then $\widetilde{T}$ preserves the measure $\tilde{\mu}$.

We will use a similar method to the proofs seen in [19].

Proof.

Any set $E\subset\widetilde{X}$ can be decomposed as as $E=\bigcup_{(q,w)\in Q^{\prime}}E_{q,w}\times\{(q,w)\}$. Since the sets $\widetilde{T}^{-1}\left(E_{q,w}\times\{(q,w)\}\right)$ are all disjoint, if we can show that $\widetilde{T}$ preserves the $\tilde{\mu}$-measure of sets of the form $E_{q,w}\times\{(q,w)\}$ then it will follow that $\widetilde{T}$ preserves the $\tilde{\mu}$-measure of $E$, and we are done.

To prove this, consider the inverse branches $T^{-1}_{a}$, $a\in A$, of $T$, such that $T^{-1}_{a}x=ax$. We may then likewise decompose $\widetilde{T}^{-1}$ into branches $\widetilde{T}^{-1}_{a}$, $a\in A$, such that $\widetilde{T}^{-1}_{a}$ induces the branch $T^{-1}_{a}$ in the first coordinate. We may then analyze exactly how $\widetilde{T}^{-1}_{a}$ acts: in particular, $\widetilde{T}^{-1}_{a}(x,(q,w))$ equals

[TABLE]

where $\delta^{-1}(q,a)$ is the unique state $p\in Q$ such that $\delta(p,a)=q$. (Unique due to $\langle Q,A,\delta,F\rangle$ being a group automaton.)

Since $\mu(aE_{q,w})=\mu(E_{q,w})/(\#A)$ and since for any subset $Y\subset X$ we have that $\tilde{\mu}(Y\times\{(q,w)\})=\mu(Y)/(\#Q^{\prime})$, we therefore have that $\tilde{\mu}(\widetilde{T}^{-1}_{a}(E_{q,w}\times\{(q,w)\}))$ equals

[TABLE]

By summing over all the $a$’s, we see that

[TABLE]

in all cases, which completes the proof. ∎

Second proof of Theorem 2.

Let $x=a_{1}a_{2}a_{3}\dots\in X$ be normal and let $y=x\upharpoonright\mkern-4.0mu\upharpoonright L=b_{1}b_{2}b_{3}\dots$. Let $\langle Q,A,\delta,I,F\rangle$ denote a group automaton that accepts $L$.

Consider any finite word $w\in A^{*}$ with length $k=|w|$. By the Pyatetskii-Shapiro normality criterion, we want to show that there exists a uniform $c>0$ (independent of our choice of $w$) such that

[TABLE]

We can analyze how often $w$ appears in $y$ by analyzing the behavior of $x$ when lifted to a augmented system with the $k$-digit buffer $\langle Q_{k},A,\delta_{k},I_{k},F_{k}\rangle$. The particular lift we choose is $\tilde{x}=(x,(q_{0},w_{0}))$. Let $i_{1},i_{2},i_{3},\dots$ be the increasing sequence of indices $i$ such that $\widetilde{T}^{i}\tilde{x}\in X\times F_{k}$, starting with the $k$th such index. Then if we let $\pi:\widetilde{X}\to A^{k}$ be the projection onto the length-$k$ word contained in the second coordinate, then we see that $y[j..j+k-1]=\pi(\widetilde{T}^{i_{j}}\tilde{x})$.

By Lemma 8, we know there is a subset $Q^{\prime}_{k}\subseteq Q_{k}$ such that eventually the orbit $\widetilde{T}^{i}\tilde{x}$ will always be in $X\times Q^{\prime}_{k}$. By, as necessary, ignoring a finite piece of $x$, we may assume that we are always in $X\times Q^{\prime}_{k}$. Since by Lemma 9, $\widetilde{T}$ preserves the measure $\tilde{\mu}$, it must also preserve the measure of $\tilde{\mu}$ when restricted to $X\times Q^{\prime}_{k}$. Therefore by Theorem 7, $\widetilde{T}$ restricted to $X\times Q^{\prime}_{k}$ is ergodic and invariant with respect to the restriction of $\tilde{\mu}$.

Let $\widetilde{C}_{w}$ denote the subset of $Q^{\prime}_{k}\cap F_{k}$ such that the second coordinate is $w$. By the last part of Lemma 8, this is always non-empty.

Then, with this definition, we have that

[TABLE]

We specified $X\times\widetilde{C}_{w}$ rather than $\pi^{-1}(w)$ so that the numerator is forced to only count among those indices $i_{j}$ rather than among all indices $i$.

According to Theorem 7, $\tilde{x}$ is normal with respect to the restriction of $\widetilde{T}$ to $X\times Q^{\prime}_{k}$. Therefore as $m$ tends to infinity we have that

[TABLE]

converges to $\tilde{\mu}(X\times\widetilde{C}_{w})/\tilde{\mu}(X\times Q^{\prime}_{k})$. Likewise, since we can write $m$ as $\#\{0\leq i\leq i_{m}-1:\widetilde{T}^{i}\tilde{x}\in X\times(Q^{\prime}_{k}\cap F_{k})\}$, we have that as $m$ tends to infinity, $m/i_{m}$ converges to $\tilde{\mu}(X\times(Q^{\prime}_{k}\cap F_{k}))/\tilde{\mu}(X\times Q^{\prime}_{k})$. Thus,

[TABLE]

But, by construction, for any set $E\subset Q_{k}$, we have that $\tilde{\mu}(X\times E)=\#E/\#Q_{k}$. Thus, this limsup is equal to

[TABLE]

We want to obtain a crude upper bound on this last fraction. We know that $\widetilde{C}_{w}\subseteq F\times\{w\}$, so $\#\widetilde{C}_{w}\leq\#F$. Moreover, by the final part of Lemma 8, we know that $\#(Q^{\prime}_{k}\cap F_{k})\geq b^{k}$. Thus,

[TABLE]

Setting $c=\#F$ completes the proof. ∎