LTI Stochastic Processes: a Behavioral Perspective

TL;DR

This paper redefines LTI stochastic processes from a behavioral perspective, providing a canonical representation and analyzing spectral density invariance, thereby deepening theoretical understanding of process interconnections.

Contribution

It introduces a behavioral framework for LTI stochastic processes, including a canonical form and a physically grounded interconnection concept, extending prior theoretical work.

Findings

Derived a canonical representation of LTI stochastic processes.

Established a notion of interconnection grounded in physical principles.

Analyzed invariance properties of spectral density distances.

Abstract

This paper revisits the definition of linear time-invariant (LTI) stochastic process within a behavioral systems framework. Building on [Willems, 2013], we derive a canonical representation of an LTI stochastic process and a physically grounded notion of interconnection between independent stochastic processes. We use this framework to analyze the invariance properties enjoyed by distances between spectral densities of LTI processes.

Equations

$$R_{\ell }w(t+\ell )+R_{\ell +1}w(t+\ell +1)+\cdots +R_{L}w(t+L)=0,$$

$$R(\sigma )w(t)=0.$$

$${\mathrm{Ker}}_{\infty }R:=\{w\in ({\mathbb{R}}^n{)}^{\mathbb{Z}}:R(\sigma )w=0\}.$$

$$R(\sigma )w(t)=e(t),$$

$$P(E):=P_e(R[E]),$$

$$P{|}_I(E):=P\left({\bigcup }_{G_i\in {\pi }^{-1}[E]}{G}_i\right),$$

$$P_1(E_1)P_2(E_2)=P_1(E_1^{\prime })P_2(E_2^{\prime }).$$

$$E_1=E_1^{\prime }\text{ and }{E}_2=E_2^{\prime }.$$

$${\Sigma }_1\wedge {\Sigma }_2:=(\mathbb{T},\mathbb{W},\mathcal{E},P),$$

$$P(E_1\cap E_2):=P_1(E_1)P_2(E_1)$$

$$\mathrm{rk}\left[\begin{array}{c}{R}_1\\ R_2\end{array}\right]=m+p.$$

$${\mathfrak{L}}_{1\wedge 2}:={\mathfrak{L}}_1\cap {\mathfrak{L}}_2={\mathrm{Ker}}_{\infty }\left[\begin{array}{c}{R}_1\\ R_2\end{array}\right].$$

$$\left[\begin{array}{cc}\sigma +a_1& \sigma +b_1\end{array}\right]w_1(t)$$

$$\left[\begin{array}{cc}\sigma +b_2& \sigma +a_2\end{array}\right]w_2(t)$$

$$\mathrm{rk}\left[\begin{array}{cc}z+a_1& z+b_1\\ z+b_2& z+a_2\end{array}\right]=2,$$

$$\left[\begin{array}{cc}\sigma +a_1& \sigma +b_1\\ \sigma +b_2& \sigma +a_2\end{array}\right]w(t)=e(t),$$

$${\Sigma }_1:R_1(\sigma )w_1(t)$$

$${\Sigma }_2:R_2(\sigma )w_2(t)$$

$${\Phi }_1(z):=R_1^{-1}(z)R_1^{-\ast }(z),{\Phi }_2(z):=R_2^{-1}(z)R_2^{-\ast }(z),$$

$$R_1^{-1}(z)=R_2^{-1}(z)V(z),$$

$$W_1(z)=W_2(z)V(z),$$

$$\mathcal{W}:=\{W\in \mathbb{R}(z{)}^{n\times n}:$$

$$W,W^{-1}\text{ analytic in }\mathcal{D}\}.$$

$$d({\Phi }_1,{\Phi }_2)$$

$$=d(W_1{V}_1{V}_1^{\ast }{W}_1^{\ast },W_2{V}_2{V}_2^{\ast }{W}_2^{\ast }),$$

$$d({\Phi }_1,{\Phi }_2)=d({\alpha }_1{\Phi }_1,{\alpha }_2{\Phi }_2),{\alpha }_1,{\alpha }_2>0.$$

$$R(\sigma )w(t)=e(t),$$

$$R(\sigma ):=\left[\begin{array}{c}{R}_1(\sigma )\\ R_2(\sigma )\end{array}\right]$$

Full text

††thanks: The research leading to these results has received funding from the European Research Council under the Advanced ERC Grant Agreement Switchlet n. 670645.

1 Introduction

Stochastic models play a crucial role in many areas of the natural and engineering sciences. Indeed, mathematical models of stochastic phenomena are widely used in many branches of physics (van Kampen, 2007), engineering (Åström, 1970; Lindquist and Picci, 2015), economics (Dupačová et al., 2002), and biology (Wilkinson, 2011). They aim at accounting for the fact that physical phenomena are unavoidably corrupted by some exogenous source of noise. This source of stochasticity sometimes represents an essential part of their model. In this paper, we address the problem of modelling stochastic phenomena from an open systems viewpoint. A system is open if it can interact with its external environment. As opposed to closed systems, open systems are amenable to interconnection. The term interconnection here is intended in the most general sense, i.e. as variable sharing between systems (Willems, 2007). In this setting, a theory of open systems is de facto a theory of interconnected systems. When dealing with deterministic linear time-invariant (LTI, for short) systems, an elegant and mature theory of open systems is provided by the theory of behaviors (Willems and Polderman, 1997). The ultimate paper of Willems is a first step towards generalizing the latter theory to a stochastic framework. Willems (2013) focuses on static stochastic systems.

Building on Willems (2013), in this paper, we treat the dynamical case. We then specialize it to the case of discrete-time LTI stochastic dynamical systems or, equivalently, LTI stochastic processes. In particular, we analyze: (i) a representation for such processes in terms of a linear model, (ii) in what sense and under which conditions these processes can be interconnected, and (iii) the invariance properties naturally inherited by distances between these processes. Interestingly, once translated into the frequency domain, these invariance properties suggest to look for a natural class of projective-like distances between rational discrete-time spectral densities.

Paper structure. The rest of the paper is organized as follows. After reviewing in §2 some results of behavioral theory, in §3 we introduce the behavioral definition of stochastic process. In §4 we focus on the linear-time invariant case. §5 considers interconnection of LTI processes. In §6 we study the invariance properties of distances between spectral densities of LTI processes. Finally, §7 collects some concluding remarks and future research directions.

*Notation. * Throughout the paper, we denote by $\mathbb{Z}$, $\mathbb{R}$, $\mathbb{R}_{+}$, and $\mathbb{C}$ the set of integer, real, non-negative real, and complex numbers, respectively. Given an element $A$ belonging to a set $\mathbb{A}$, $A^{\mathrm{c}}$ will denote the complement of $A$ w.r.t. $\mathbb{A}$, while $\emptyset$ the empty set. We let $\mathbb{R}^{n\times m}$ denote the set of real-valued $n\times m$ matrices and $A^{\top}\in\mathbb{R}^{m\times n}$ denote the transpose of $A\in\mathbb{R}^{n\times m}$. The symbols $\mathbb{R}[z,z^{-1}]^{n\times m}$ and $\mathbb{R}(z)^{n\times m}$ stand for the set of $n\times m$ Laurent polynomial matrices and the set of $n\times m$ rational matrices with real coefficients in the indeterminate $z$, respectively. The normal rank of a rational matrix $A(z)\in\mathbb{R}(z)^{n\times m}$ is defined as the rank of $A(z)$ almost everywhere in $z\in\mathbb{C}$ and it will be denoted by $\mathrm{rk}(A)$. Given $A(z)\in\mathbb{R}(z)^{n\times m}$, we let $A^{*}(z):=A^{\top}(1/z)$ and, for square $A(z)$’s of full normal rank, $A^{-*}(z):=[A^{\top}(1/z)]^{-1}$. We recall that a Laurent unimodular matrix is a square Laurent polynomial matrix whose inverse is also Laurent polynomial or, equivalently, whose determinant is a non-zero monomial $\lambda z^{k}$, $\lambda\in\mathbb{R}\setminus\{0\}$, $k\in\mathbb{Z}$. We denote by $\mathbb{U}[z,z^{-1}]^{n\times n}$ the group of $n\times n$ (Laurent) unimodular matrices and by $\mathcal{S}_{+}(\mathbf{T})^{n\times n}$ the set of $n\times n$ matrix-valued functions which are positive definite on the unit circle $\mathbf{T}:=\{\,z\in\mathbb{C}\,:\,|z|\ =1\,\}$, i.e. $n\times n$ discrete-time coercive spectral densities. $\mathcal{S}_{\mathrm{rat}}^{n\times n}\subset\mathcal{S}_{+}(\mathbf{T})^{n\times n}$ will denote the set of $n\times n$ rational discrete-time coercive spectral densities. Finally, we shall suppose the reader to be acquainted with some elementary notions of probability theory, e.g. the definitions of $\sigma$-algebra, (smallest) $\sigma$-algebra generated by a collection of sets, Borel $\sigma$-algebra, probability measure; notions that can be found in any standard textbook of probability or measure theory, e.g. Billingsley (1986).

2 background on behavioral theory

In this preliminary section, we quickly review some basic notions and results of behavioral theory. We refer the reader to the seminal papers Willems (1986, 1989, 1991) and to the monograph Willems and Polderman (1997) for a comprehensive treatment on the subject.

In the theory of behaviors, a dynamical system is defined as a triple $\Sigma:=(\mathbb{T},\mathbb{W},\mathfrak{B})$, where $\mathbb{T}$ is the set of times over which the system evolves (time axis), $\mathbb{W}$ is the set over which the variables of the signals being modelled take values (signal space), and $\mathfrak{B}$ is a subset of $\mathbb{W}^{\mathbb{T}}$ (i.e. the set of all maps from $\mathbb{T}$ to $\mathbb{W}$, also called universum) in which all the admissible system trajectories live (the behavior of the system). The dynamical system $\Sigma=(\mathbb{T},\mathbb{W},\mathfrak{B})$ is linear if $\mathbb{W}$ is a vector space and $\mathfrak{B}$ a linear subspace of $\mathbb{W}^{\mathbb{T}}$. $\Sigma$ is said to be time-invariant if $\mathbb{T}$ is closed under addition and $\sigma^{t}\mathfrak{B}\subseteq\mathfrak{B}$ for all $t\in\mathbb{T}$, where $\sigma$ denotes the backward shift operator defined as $(\sigma f)(t^{\prime}):=f(t^{\prime}+1)$. In this paper we mainly focus on $n$-dimensional, real-valued, discrete-time systems. Hence we set $\mathbb{T}:=\mathbb{Z}$ and $\mathbb{W}:=\mathbb{R}^{n}$. As a consequence, the behavior of the system becomes a subset of $(\mathbb{R}^{n})^{\mathbb{Z}}$ (the set of maps from $\mathbb{Z}$ to $\mathbb{R}^{n}$), i.e. a family of $n$-dimensional, real-valued, discrete-time sequences $w\in(\mathbb{R}^{n})^{\mathbb{Z}}$. An $n$-dimensional LTI system may be described by an Auto-Regressive (AR) model111We remark that the behavior is always deterministic, i.e. it is composed by deterministic trajectories. Therefore, in this section, the term AR does not refer to stochastic framework.

[TABLE]

for all $t\in\mathbb{Z}$, where $R_{i}\in\mathbb{R}^{p\times n}$, $i=\ell,\ell+1,\dots,L$, $\ell,L\in\mathbb{Z},\,L>\ell$. The Laurent polynomial matrix $R(z):=R_{\ell}z^{\ell}+R_{\ell+1}z^{\ell+1}+\dots+R_{L}z^{L}\in\mathbb{R}[z,z^{-1}]^{p\times n}$ defines an operator in the shift $\sigma$, $R\colon(\mathbb{R}^{n})^{\mathbb{Z}}\to(\mathbb{R}^{p})^{\mathbb{Z}}$ which allows to rewrite the previous expression as

[TABLE]

The behavior of the LTI system is then given by

[TABLE]

Equation (1) is known as the kernel representation of an LTI behavior. An LTI behavior $\mathfrak{B}\subseteq(\mathbb{R}^{n})^{\mathbb{Z}}$ is said to be complete if for all $w(t)\in\mathfrak{B}$, $\left.w\right|_{[t_{1},t_{2}]}\in\left.\mathfrak{B}\right|_{[t_{1},t_{2}]}$, $\forall\,t_{1},t_{2}\in\mathbb{Z}$, $t_{1}\leq t_{2}$, where $\left.w\right|_{[t_{1},t_{2}]}$ and $\left.\mathfrak{B}\right|_{[t_{1},t_{2}]}$ denote the restriction of $w(t)$ and $\mathfrak{B}$, respectively, to the time interval $[t_{1},t_{2}]$. A fundamental result in behavioral theory states that every LTI complete behavior admits a kernel representation (1).

To conclude, consider two LTI complete behaviors $\mathfrak{B}_{1}$, $\mathfrak{B}_{2}$ with kernel representations described by $R_{1}(z)$, $R_{2}(z)\in\mathbb{R}[z,z^{-1}]^{p\times n}$, respectively. We say that the two behaviors $\mathfrak{B}_{1}$, $\mathfrak{B}_{2}$ are equivalent if $R_{1}(z)=U(z)R_{2}(z)$ with $U(z)\in\mathbb{U}[z,z^{-1}]^{p\times p}$ being a Laurent unimodular matrix. Hence, with reference to the kernel representation, every behavior is uniquely determined by its kernel matrix up to a unimodular transformation acting on the left.

3 From deterministic to                            stochastic behaviors

Using the terminology of Willems (2013), a stochastic system is defined as a triple $(\mathbb{V},\mathscr{E},P)$, where $\mathbb{V}$ is the outcome space, $\mathscr{E}$ is a $\sigma$-algebra of events, and $P\colon\mathscr{E}\to[0,1]$ is a probability measure which assigns to each event in $\mathscr{E}$ a value in the interval $[0,1]$.

Consider the deterministic system $\Sigma=(\mathbb{T},\mathbb{W},\mathfrak{B})$. $\Sigma$ can be regarded as a very special stochastic system. As a matter of fact, $\Sigma$ coincides with the stochastic system $(\mathbb{W}^{\mathbb{T}},\{\emptyset,\mathbb{W}^{\mathbb{T}},\mathfrak{B},\mathfrak{B}^{\mathrm{c}}\},P_{\Sigma})$ in which $\mathfrak{B}$ is a certain event, meaning that $P_{\Sigma}(\mathfrak{B})=1$. Since the previous definition unavoidably involves the probability measure $P_{\Sigma}$ (and specifically the constraint $P_{\Sigma}(\mathfrak{B})=1$), we could clearly have used a $\sigma$-algebra of events richer than $\{\emptyset,\mathbb{W}^{\mathbb{T}},\mathfrak{B},\mathfrak{B}^{\mathrm{c}}\}$. However the latter seems to be a more natural choice due to the fact that it is the most “parsimonious” $\sigma$-algebra, that is, the smallest possible $\sigma$-algebra of events containing the deterministic behavior $\mathfrak{B}$.

Now assume that some source of stochasticity is added to the deterministic system $\Sigma$ (for instance, some additive noise acting on the trajectories of $\mathfrak{B}$), then we expect that the newly generated (stochastic) system will possess a richer $\sigma$-algebra of events—indeed the noise modifies, and, more precisely, enlarges the space of admissible trajectories of the system—and, as a consequence, a new probability measure. Furthermore, by adding more and more sets to our event space, we are, in a sense, moving more and more away from the class of deterministic systems. Loosely speaking, the cardinality of the $\sigma$-algebra of events can be considered as a measure of the “degree” of stochasticity of the system.

From this intuitive description, we can see that the $\sigma$-algebra of events plays an important role in the mathematical model of a stochastic system, perhaps as important as the probability measure associated to the system. When dealing with static systems, i.e. systems which do not evolve in time, this is exactly the point raised in Willems (2013). One of the aims of the present paper is to extend this viewpoint to the dynamical case. To this end, we first revisit the definition of stochastic process in the spirit of behavioral theory.

Definition 1

A stochastic process is a quadruple $\Sigma:=(\mathbb{T},\mathbb{W},\mathscr{E},P)$, where

$\mathbb{T}$* is the time axis,* 2. 2.

$\mathbb{W}$* is the signal space, i.e. the set in which the variables whose (noisy) time evolution is modelled take on their values,* 3. 3.

$\mathscr{E}$* is a $\sigma$-algebra of subsets of $\mathbb{W}^{\mathbb{T}}$ with elements called events,* 4. 4.

$P\colon\mathscr{E}\to[0,1]$* is the probability measure defined on the $\sigma$-algebra of events.*

With reference to the above definition, we observe that:

1. (i)

A stochastic process is a probability space where the outcome space is given by $\mathbb{W}^{\mathbb{T}}$. Two important classes of stochastic processes are obtained by selecting $\mathbb{T}=\mathbb{R}$, in which case the outcome space is the space of functions $f\colon\mathbb{R}\to\mathbb{W}$, and $\mathbb{T}=\mathbb{Z}$, in which case the outcome space is the space of sequences $\{f_{t}\}_{t\in\mathbb{Z}}$ taking values on $\mathbb{W}$. Intuitively, we can think of a stochastic process as a system described by a collection of “behaviors” with assigned probabilities, where $P$ specifies the probability of each “behavior”. 2. (ii)

The standard definition of stochastic process is a family of random variables (i.e. measurable functions) $\{f_{t}\}_{t\in\mathbb{T}}$ defined on some probability space and parametrized by an index $t\in\mathbb{T}$, which usually represents time. By specifying the finite-dimensional probability distributions of the family $\{f_{t}\}_{t\in\mathbb{T}}$ it is then possible to characterize the infinite-dimensional distributions of the process (by virtue of Kolmogorov existence theorem (Billingsley, 1986, Thm. 36.1)). Our definition of a stochastic process is essentially equivalent to the latter one but formulated in terms of $\sigma$-algebras of events defined on the (usually infinite-dimensional) space of trajectories $\mathbb{W}^{\mathbb{T}}$. From this point of view, in Definition 1 emphasis is put on the event space itself rather than on the variables that generate that space.

For the rest of the paper, we restrict our attention to the class of $n$-dimensional, real-valued, discrete-time stochastic processes, i.e. we set $\mathbb{T}=\mathbb{Z}$ and $\mathbb{W}=\mathbb{R}^{n}$. Notice that, in this setting, we can identify two particular subclasses of stochastic processes, namely the subclass of deterministic dynamical systems whose $\sigma$-algebra of events is given by $\{\emptyset,(\mathbb{R}^{n})^{\mathbb{Z}},\mathfrak{B},\mathfrak{B}^{\mathrm{c}}\}$ with $\mathfrak{B}\subset(\mathbb{R}^{n})^{\mathbb{Z}}$, and the subclass of classical stochastic processes whose $\sigma$-algebra of events is given by the Borel $\sigma$-algebra generated by the open sets of $(\mathbb{R}^{n})^{\mathbb{Z}}$ equipped with the product topology (i.e. the topology of pointwise convergence), which we will denote by $\mathscr{B}((\mathbb{R}^{n})^{\mathbb{Z}})$. Since the Borel $\sigma$-algebra of $(\mathbb{R}^{n})^{\mathbb{Z}}$ coincides with the $\sigma$-algebra containing all the non-pathological subsets of $(\mathbb{R}^{n})^{\mathbb{Z}}$, we can think of these two subclasses as two extremes in the space of stochastic processes.

Remark 2

It is worth noting that when dealing with continuous-time stochastic processes ($\mathbb{T}=\mathbb{R}$) the Borel $\sigma$-algebra generated by the open sets of the product space $(\mathbb{R}^{n})^{\mathbb{R}}$ equipped with the product topology proves often to be inadequate for describing the events of the process (Billingsley, 1986, Ch. 7). Indeed, for instance, it can be shown that the many “interesting” sets of functions, e.g. the set of continuous functions, are not contained in $\mathscr{B}((\mathbb{R}^{n})^{\mathbb{R}})$. To overcome this issue, other types of $\sigma$-algebras can be considered in place of $\mathscr{B}((\mathbb{R}^{n})^{\mathbb{R}})$, for instance the Borel $\sigma$-algebra generated by the open sets in the space of continuous functions equipped the topology of uniform convergence on compact sub-intervals. $\diamondsuit$

4 Linear Time-Invariant                  stochastic processes

In this section we introduce the notion of linear time-invariant (discrete-time) stochastic process. The events of these processes are uniquely defined up to trajectories belonging to a deterministic LTI behavior. We then discuss a canonical representation for these systems.

Definition 3

The stochastic process $\Sigma:=(\mathbb{Z},\mathbb{R}^{n},\mathscr{E},P)$ is said to be linear and time-invariant (LTI, for short) if there exists a linear and time-invariant behavior $\mathfrak{L}\subset(\mathbb{R}^{n})^{\mathbb{Z}}$ such that the events are the Borel subsets of the quotient space $(\mathbb{R}^{n})^{\mathbb{Z}}/\mathfrak{L}$, i.e. $\mathscr{E}:=\mathscr{B}((\mathbb{R}^{n})^{\mathbb{Z}}/\mathfrak{L})$, and $P$ is a Borel probability measure on the same quotient space, i.e. $P\colon\mathscr{B}((\mathbb{R}^{n})^{\mathbb{Z}}/\mathfrak{L})\to[0,1]$.

Observe that $\mathscr{B}((\mathbb{R}^{n})^{\mathbb{Z}}/\mathfrak{L})$ is a well-defined Borel $\sigma$-algebra. Indeed, it coincides exactly with the Borel $\sigma$-algebra generated by the open sets $A/\mathfrak{L}$ (open sets of the quotient topology), with $A$ an open set of the topological vector space $(\mathbb{R}^{n})^{\mathbb{Z}}$ equipped with the product topology. Moreover, using the terminology introduced in Willems (2013), we call $\mathfrak{L}$ the fiber of the LTI stochastic process.

Definition 3 can be intuitively interpreted as follows: given any (Borel) subset $\bar{E}\subset(\mathbb{R}^{n})^{\mathbb{Z}}$, which consists of a subset of trajectories in $(\mathbb{R}^{n})^{\mathbb{Z}}$, if the stochastic process $\Sigma$ is LTI with fiber $\mathfrak{L}$ then the subset $E$ generated by adding to $\bar{E}$ the trajectories belonging to the LTI behavior $\mathfrak{L}$ is an event of $\Sigma$. Loosely speaking, an event is a collection of subsets in $(\mathbb{R}^{n})^{\mathbb{Z}}$, where each subset contains trajectories “parallel” to the LTI behavior $\mathfrak{L}$ (see Fig. 1 for an illustration).

Throughout the paper, we will often make the assumption that fibers are complete LTI behaviors, this means that all the trajectories belonging to the fiber admit a finite dimensional characterization, that is, they are uniquely determined by their restrictions over all possible finite time intervals. Under this assumption, the fiber of an LTI stochastic process can always be represented by means of the kernel of a Laurent polynomial matrix, as recalled in §2. LTI processes characterized by a complete fiber admit a canonical representation, called kernel representation by analogy with the deterministic case.

Theorem 4

A stochastic process $\Sigma=(\mathbb{Z},\mathbb{R}^{n},\mathscr{E},P)$ is an LTI stochastic process with fiber $\mathfrak{L}$ being a complete LTI behavior if and only if $\Sigma$ can be described by a stochastic sequence $w(\cdot)$ satisfying for all $t\in\mathbb{Z}$

[TABLE]

where $R(z)\in\mathbb{R}[z,z^{-1}]^{m\times n}$ is of full row normal rank, i.e. $\mathrm{rk}(R)=m$, and $e(\cdot)$ describes the stochastic process $\Sigma_{e}:=(\mathbb{Z},\mathbb{R}^{m},\mathscr{B}((\mathbb{R}^{m})^{\mathbb{Z}}),P_{e})$.

{pf}

See Appendix A.

With reference to the previous result, we remark that if $E\in\mathscr{E}$ is an event of $\Sigma$ then its probability measure $P$ is defined through $e(\cdot)$ by

[TABLE]

being $R[E]\in\mathscr{B}((\mathbb{R}^{m})^{\mathbb{Z}})$ the image of $E$ under $R(\sigma)$.

Furthermore, if we consider the restriction of an LTI process to a finite set of time indices, say $I:=\{t_{1},t_{2},\dots,t_{n}\}$, $t_{i}\in\mathbb{Z}$, $n>0$, then we obtain a (static) stochastic system described by the triple $\Sigma|_{I}:=((\mathbb{R}^{n})^{|I|},\mathscr{E}|_{I},P|_{I})$, where $|I|$ is the cardinality of the set $I$, $\mathscr{E}|_{I}:=\mathscr{B}((\mathbb{R}^{n})^{|I|}/\mathfrak{L}|_{I})$ with $\mathfrak{L}|_{I}$ the restriction of the complete LTI behavior to the time set $I$, and $P|_{I}$ a restricted probability measure defined for all $E\in\mathscr{E}|_{I}$ as

[TABLE]

being $\pi^{-1}[E]$ the pre-image of $E$ under the canonical projection $\pi\colon(\mathbb{R}^{n})^{\mathbb{Z}}\to(\mathbb{R}^{n})^{|I|}$, $\{f_{t}\}_{t\in\mathbb{Z}}\mapsto\{f_{t_{1}},f_{t_{2}},\dots,f_{t_{n}}\}$. Since, in general, the restriction $\mathfrak{L}|_{I}$ returns a non-empty linear finite-dimensional subspace of $(\mathbb{R}^{n})^{|I|}$, we note that $\Sigma|_{I}$ does not, in general, describe a classical random vector, where for classical we mean a random vector characterized by a Borel $\sigma$-algebra of events on $(\mathbb{R}^{n})^{|I|}$, as in (Willems, 2013, Def. 2).

5 interconnection of                            stochastic processes

Interconnection is a property characterizing open systems, i.e. systems which are allowed to interact with their environment. With reference to the mathematical model of a deterministic dynamical system, this interaction can take place if some variables of the system are left unmodelled (Willems, 2007). In this section we present an extension of the definition of interconnection between deterministic dynamical systems which applies to stochastic processes. After introducing some general definitions, we will focus on the discrete-time LTI case.

As in the deterministic case, interconnection of two stochastic processes can be thought of as variable sharing between the two processes. In other words, interconnection between two processes is obtained by simply imposing an equality constraint on the variables describing the stochastic laws of the two processes (Fig. 2).

In the deterministic case, given two dynamical systems $\Sigma_{1}=(\mathbb{W},\mathbb{T},\mathfrak{B}_{1})$ and $\Sigma_{2}=(\mathbb{W},\mathbb{T},\mathfrak{B}_{2})$ having the same time axis and signal space, the interconnection between $\Sigma_{1}$ and $\Sigma_{2}$ is defined as the deterministic system $\Sigma_{1}\wedge\Sigma_{2}:=(\mathbb{W},\mathbb{T},\mathfrak{B}_{1}\cap\mathfrak{B}_{2})$ (Willems, 2007). In the stochastic case, the definition of interconnection we are going to present is similar to the latter one if we replace the role of the deterministic behaviors with the $\sigma$-algebras of events of the processes (which indeed represent collections of admissible “behaviors” of the processes). However, in this case, a problem arises. As a matter of fact, since interconnection of stochastic processes also involves the probability laws defined on the processes, a natural compatibility condition between the two to-be-interconnected processes has to be fulfilled. This natural condition states that the probability measure defined on the interconnected process must be consistent, in a sense explained below, with the probability measures defined on the original processes. This condition was introduced in Willems (2013) with reference to (static) stochastic systems under the name of complementarity. In the following definition, we adapt the notion of complementarity to the case of stochastic processes.

Definition 5

Two stochastic processes $\Sigma_{1}=(\mathbb{T},\mathbb{W},\mathscr{E}_{1},P_{1})$ and $\Sigma_{2}=(\mathbb{T},\mathbb{W},\mathscr{E}_{2},P_{2})$ are said to be complementary if for all $E_{1}$, $E_{1}^{\prime}\in\mathscr{E}_{1}$ and $E_{2}$, $E_{2}^{\prime}\in\mathscr{E}_{2}$ such that $E_{1}\cap E_{2}=E_{1}^{\prime}\cap E_{2}^{\prime}$ it holds

[TABLE]

Moreover, the two $\sigma$-algebras $\mathscr{E}_{1}$ and $\mathscr{E}_{2}$ are said to be complementary if for all non-empty sets $E_{1}$, $E_{1}^{\prime}\in\mathscr{E}_{1}$ and $E_{2}$, $E_{2}^{\prime}\in\mathscr{E}_{2}$ such that $E_{1}\cap E_{2}=E_{1}^{\prime}\cap E_{2}^{\prime}$ it holds

[TABLE]

Remark 6

The notion of complementarity between $\sigma$-algebra of the processes is weaker than the notion of complementarity of processes. Indeed, the former represents only a sufficient condition for complementarity of processes. However, working with complementarity of $\sigma$-algebra is usually easier since this notion does not involve the probability laws describing the processes, as pointed out also in Willems (2013). $\diamondsuit$

Under the assumption of complementarity between two stochastic processes, we arrive at a formal definition of interconnection.

Definition 7

Let $\Sigma_{1}=(\mathbb{T},\mathbb{W},\mathscr{E}_{1},P_{1})$ and $\Sigma_{2}=(\mathbb{T},\mathbb{W},\mathscr{E}_{2},$ $P_{2})$ be two independent222We say that two stochastic processes are (stochastically) independent if their $\sigma$-algebras of events are so, with respect to any joint probability measure. and complementary stochastic processes. The interconnection of $\Sigma_{1}$ and $\Sigma_{2}$ is defined as the stochastic process

[TABLE]

where $\mathscr{E}$ is the $\sigma$-algebra generated by $\mathscr{E}_{1}\cup\mathscr{E}_{2}$ and $P$ is defined on the sets $\{E_{1}\cap E_{2}\,:\,E_{1}\in\mathscr{E}_{1},E_{2}\in\mathscr{E}_{2}\}$ by

[TABLE]

and extended to all of $\mathscr{E}$ by virtue of the Hahn-Kolmogorov extension theorem (Billingsley, 1986, Ch. 3).333The Hahn-Kolmogorov theorem gives conditions under which a function $\mu\colon\mathscr{A}\to[0,1]$ defined on an algebra $\mathscr{A}$ of subsets of $\Omega$ can be extended to a unique bona fide probability measure on the $\sigma$-algebra generated by $\mathscr{A}$. These conditions are: (i) $\mu(\Omega)=1$, (ii) countably additivity, i.e. $\mu\left(\bigcup_{i=1}^{\infty}A_{i}\right)=\sum_{i=1}^{\infty}\mu(A_{i})$ for any countable disjoint family of subsets $\{A_{i}\}_{i=1}^{\infty}$, $A_{i}\in\mathscr{A}$, such that $\bigcup_{i=1}^{\infty}A_{i}\in\mathscr{A}$. (A function satisfying these two requirements is called a pre-measure on $\mathscr{A}$.) In our case, the theorem applies to $P$ since the latter is defined through the product of two bona fide probability measures.

Remark 8

It is worth pointing out that interconnection between stochastic processes, as given in Definition 7, differs from the classical notion of coupling between stochastic processes. As a matter of fact, even in the static case, coupling of two stochastic systems $\Sigma_{1}=(\mathbb{W},\mathscr{E}_{1},P_{1})$ and $\Sigma_{2}=(\mathbb{W},\mathscr{E}_{2},P_{2})$ requires the construction of a new stochastic system with signal space $\mathbb{W}\times\mathbb{W}$, $\sigma$-algebra generated by the sets $\{E_{1}\times E_{2}\,:\,E_{1}\in\mathscr{E}_{1},\,E_{2}\in\mathscr{E}_{2}\}$, and probability measure having prescribed marginal distributions, see e.g. Lindvall (2002). On the other hand, interconnection between $\Sigma_{1}$ and $\Sigma_{2}$ means that a new $\sigma$-algebra is constructed on the same signal space shared by the two to-be-interconnected systems, that is $\mathbb{W}$. More precisely, the events which lie in the intersection between the two $\sigma$-algebras $\mathscr{E}_{1}$ and $\mathscr{E}_{2}$ generate the $\sigma$-algebra of the interconnected system. From this viewpoint, coupling appears more similar to juxtaposition of stochastic processes than interconnection, where for juxtaposition we mean that starting from two processes described by stochastic laws $w_{1}$ and $w_{2}$ we construct a new process described by $(w_{1},w_{2})$, as in Willems (2013). We can therefore think of interconnection as a restriction in the signal space and coupling as an expansion of the signal space. $\diamondsuit$

We now restrict the attention to the class of LTI stochastic processes. In this case, the fiber of the process is given by an LTI behavior. If we add the further assumption that the fiber is described by a complete LTI behavior, then, by virtue of Theorem 4, the process admits a kernel representation. For this class of stochastic processes, it is possible to derive a condition on the kernel matrices which is equivalent to complementary of $\sigma$-algebras of events of the processes.

Theorem 9

Consider two (stochastically) independent LTI complete stochastic processes $\Sigma_{1}:=(\mathbb{Z},\mathbb{R}^{n},\mathscr{E}_{1},P_{1})$ and $\Sigma_{2}:=(\mathbb{Z},\mathbb{R}^{n},\mathscr{E}_{2},P_{2})$ described by fibers $\mathfrak{L}_{1}:=\mathrm{Ker}_{\infty}R_{1}$ and $\mathfrak{L}_{2}:=\mathrm{Ker}_{\infty}R_{2}$, for suitable Laurent polynomial matrices $R_{1}(z)\in\mathbb{R}[z,z^{-1}]^{m\times n}$, $R_{2}(z)\in\mathbb{R}[z,z^{-1}]^{p\times n}$ with $\mathrm{rk}(R_{1})=m$ and $\mathrm{rk}(R_{2})=p$. The two $\sigma$-algebras $\mathscr{E}_{1}$ and $\mathscr{E}_{2}$ are complementary if and only if it holds

[TABLE]

In this case, the fiber of the interconnected process $\Sigma_{1}\wedge\Sigma_{2}$ is given by

[TABLE]

{pf}

See Appendix A.

Example 10

As a simple example of interconnection, consider two LTI processes $\Sigma_{1}:=(\mathbb{Z},\mathbb{R}^{2},\mathscr{E}_{1},P_{1})$ and $\Sigma_{2}:=(\mathbb{Z},\mathbb{R}^{2},\mathscr{E}_{2},P_{2})$ described by kernel representations

[TABLE]

respectively. Furthermore, assume that $e_{1}(\cdot)$ and $e_{2}(\cdot)$ describe stochastically independent processes $\Sigma_{e_{1}}=(\mathbb{Z},\mathbb{R},\mathscr{B}(\mathbb{R}^{\mathbb{Z}}),P_{e_{1}})$ and $\Sigma_{e_{2}}=(\mathbb{Z},\mathbb{R},\mathscr{B}(\mathbb{R}^{\mathbb{Z}}),P_{e_{2}})$, respectively. If we partition the variables $w_{1}$ and $w_{2}$ in a “input-output” form $w_{1}:=[u_{1}\ y_{1}]^{\top}$ and $w_{2}:=[y_{2}\ u_{2}]^{\top}$, then the two LTI processes $\Sigma_{1}$ and $\Sigma_{2}$ can be regarded as two noisy input/output LTI systems (see also Fig. 3). The two $\sigma$-algebras $\mathscr{E}_{1}$ and $\mathscr{E}_{2}$ are complementary if and only if

[TABLE]

or, equivalently, if and only if $a_{1}+a_{2}\neq b_{1}+b_{2}$ and $a_{1}a_{2}\neq b_{1}b_{2}$. If the latter conditions are met, the interconnected process $\Sigma_{1}\wedge\Sigma_{2}$ is a well-defined LTI process described by the laws of the stochastic sequence $w(t):=[w_{1}(t)\ w_{2}(t)]^{\top}$ satisfying

[TABLE]

where $e(\cdot)$ describe the process $\Sigma_{e}=(\mathbb{Z},\mathbb{R}^{2},\mathscr{B}((\mathbb{R}^{2})^{\mathbb{Z}}),P_{e})$ with $P_{e}$ defined as $P_{e}(E_{1}\times E_{2}):=P_{e_{1}}(E_{1})P_{e_{2}}(E_{2})$ for all $E_{1},\,E_{2}\in\mathscr{B}(\mathbb{R}^{\mathbb{Z}})$ and extended to all of $\mathscr{B}((\mathbb{R}^{2})^{\mathbb{Z}})$ via the Hahn-Kolmogorov extension theorem. $\diamondsuit$

To conclude this section, we present a straightforward corollary of Theorem 9 which gives a characterization of interconnected processes with a “full” $\sigma$-algebra of events.

Corollary 11

Consider the two Laurent polynomial matrices $R_{1}(z)\in\mathbb{R}[z,z^{-1}]^{m\times n}$ and $R_{2}(z)\in\mathbb{R}[z,z^{-1}]^{p\times n}$ describing the LTI processes defined in Theorem 9. The $\sigma$-algebra of the interconnected system $\Sigma_{1}\wedge\Sigma_{2}$ is given by $\mathscr{E}=\mathscr{B}((\mathbb{R}^{n})^{\mathbb{Z}})$ if and only if $R:=[R_{1}^{\top}\ R_{2}^{\top}]^{\top}\in\mathbb{U}[z,z^{-1}]^{n\times n}$, i.e. $R$ is a unimodular Laurent polynomial matrix.

6 Metric invariance properties of LTI stochastic processes

In the previous sections, we have pointed out that in the definition of stochastic process a crucial role is played by the event space $\mathscr{E}$. For LTI stochastic processes, the structure of the event space is characterized by its fiber, i.e. by the subspace of trajectories described by an LTI behavior. Let us restrict the attention to fibers defined by complete behaviors and consider two stochastic processes in their kernel representations

[TABLE]

with $R_{1}(z)\in\mathbb{R}[z,z^{-1}]^{m\times n}$, $R_{2}(z)\in\mathbb{R}[z,z^{-1}]^{m\times n}$, $\mathrm{rk}(R_{1})=\mathrm{rk}(R_{2})=m$, and $e_{1}$, $e_{2}$ describing the processes $(\mathbb{Z},\mathbb{R}^{m},\mathscr{B}((\mathbb{R}^{m})^{\mathbb{Z}}),P_{e_{1}})$, $(\mathbb{Z},\mathbb{R}^{m},\mathscr{B}((\mathbb{R}^{m})^{\mathbb{Z}}),P_{e_{2}})$, respectively. If the two processes happen to have the same fiber, then it follows that $\mathrm{Ker}_{\infty}R_{1}$ and $\mathrm{Ker}_{\infty}R_{2}$ are equivalent behaviors. This in turn implies that $R_{1}$ and $R_{2}$ are connected by a unimodular transformation acting on the left, i.e. $R_{1}(z)=U(z)R_{2}(z)$ with $U(z)\in\mathbb{U}[z,z^{-1}]^{m\times m}$. In view of this fact, we introduce the following definition of equivalence between complete LTI stochastic processes.

Definition 12

Consider two complete LTI stochastic processes $\Sigma_{1}$, $\Sigma_{2}$ described by kernel matrices $R_{1}$, $R_{2}\in\mathbb{R}[z,z^{-1}]^{m\times n}$. We say that $\Sigma_{1}$ and $\Sigma_{2}$ are equivalent if their fibers are equivalent behaviors, or, equivalently, if $R_{1}=UR_{2}$ with $U\in\mathbb{U}[z,z^{-1}]^{m\times m}$.

We now investigate what the above defined equivalence between LTI processes entails, when considering spectral densities of LTI stochastic processes. To this extent, assume that $e_{1},\,e_{2}$ are unit-variance white noise processes, and $w_{1}$, $w_{2}$ stochastic processes in the classical sense, i.e. equipped with Borel $\sigma$-algebras of events.444The reader will notice that the classical notion of stochastic process is always recovered by enriching the $\sigma$-algebra of events and suitably redefining the probability over the new $\sigma$-algebra. For $n=m$, the spectral densities of $w_{1}$ and $w_{2}$ in (6)-(7) are given, respectively, by

[TABLE]

and the fact that $\Sigma_{1}$ and $\Sigma_{2}$ are equivalent translates into the relation

[TABLE]

for a suitable unimodular matrix $V(z)\in\mathbb{U}[z,z^{-1}]^{n\times n}$. We can generalize this observation to arbitrary rational discrete-time coercive spectral densities as follows.

Definition 13

Consider two rational spectral densities $\Phi_{1},\,\Phi_{2}\in\mathcal{S}_{\mathrm{rat}}^{n\times n}$ and let $W_{1},\,W_{2}\in\mathbb{R}(z)^{n\times n}$ be spectral factors of $\Phi_{1},\,\Phi_{2}$, respectively, that is $\Phi_{i}=W_{i}W_{i}^{*}$, $i=1,2$. We say that $\Phi_{1},\,\Phi_{2}$ are unimodular equivalent if

[TABLE]

with $V(z)\in\mathbb{U}[z,z^{-1}]^{n\times n}$.

Remark 14

Notice that the above notion of unimodular equivalence depends upon the choice of the spectral factors $W_{1}$ and $W_{2}$ of $\Phi_{1}$ and $\Phi_{2}$, respectively. However, if we restrict $W_{1}$ and $W_{2}$ to belong to the set of stable spectral factors with stable inverse, i.e. spectral factors analytic in the region $\mathcal{D}:=\{\,z\in\mathbb{C}\,:\,|z|\ >1\,\}$ with inverse analytic in the same region, then this is not the case.555Here the term stable refers to the finite zero-pole structure of a spectral factor. In particular, such a spectral factor can possess zeros/poles at infinity. This follows from the fact that

1. (i)

if $W^{\prime},\,W^{\prime\prime}\in\mathbb{R}(z)^{n\times n}$ are stable spectral factors of $\Phi\in\mathcal{S}_{\mathrm{rat}}^{n\times n}$ with stable inverse, then $W^{\prime}=W^{\prime\prime}Q$ with $Q\in\mathbb{R}(z)^{n\times n}$ being an all-pass (Laurent) unimodular matrix; 2. (ii)

product of (Laurent) unimodular matrices is again (Laurent) unimodular.

In view of this, in Definition 13 we will always consider spectral factors belonging to the set

[TABLE]

In light of Definition 13 and Remark 14, a spectral density $\Phi\in\mathcal{S}_{\mathrm{rat}}^{n\times n}$ is uniquely determined by any spectral factor in $\mathscr{W}$, modulo a unimodular matrix.

At this point, it is interesting to analyze the invariance properties of distances in $\mathcal{S}_{\mathrm{rat}}^{n\times n}$, naturally induced by the unimodular equivalence relation (8). Specifically, from (8) it follows that any natural distance $d\colon\mathcal{S}_{\mathrm{rat}}^{n\times n}\times\mathcal{S}_{\mathrm{rat}}^{n\times n}\to\mathbb{R}_{+}$ must satisfy

[TABLE]

for unimodular matrices $V_{1}$, $V_{2}\in\mathbb{U}[z,z^{-1}]^{n\times n}$. We term the invariance property (9) unimodular invariance. The behavioral framework suggests that distances between spectral densities of LTI stochastic processes should be unimodular invariant.

In particular, w.r.t. the scalar case, unimodular transformations take the form $u(z)=\lambda z^{k}$, $\lambda\in\mathbb{R}\setminus\{0\}$ and $k\in\mathbb{Z}$. Therefore unimodular invariance (9) reduces to a projective invariance property. Namely, letting $\Phi_{1},\,\Phi_{2}\in\mathcal{S}_{\mathrm{rat}}^{1\times 1}$, it holds

[TABLE]

Consequently, in the scalar case, a distance which satisfies unimodular invariance (9) is a distance between “shapes” of spectral densities, since such a distance must be insensitive to scalings. Distances of this kind have been extensively investigated in the literature, see e.g. Martin (2000); Georgiou (2007a, b), and have found interesting applications in the area of speech processing. With reference to the multivariate case, it would be interesting to investigate the existence of metrics that are invariant in the sense of Eq. (9). This topic will be the subject of future investigation.

7 Conclusions and future work

In this paper, we addressed the problem of modelling stochastic dynamical systems from a behavioral perspective. We focused on LTI processes and we analyzed their interconnection and metric invariance properties. More specifically, an open-systems notion of interconnection between LTI stochastic processes and a natural invariance property enjoyed by distances in the space of rational spectral densities have been discussed. Directions for future research include: (i) the analysis of the notion of interconnection here introduced and its applications, (ii) the study of metrics in the space of rational discrete-time spectral densities featuring the invariance property presented in §6.

Appendix A Extended proofs

In this Appendix, we present the proofs of Theorem 4 and Theorem 9 of the main text.

Proof of Theorem 4.   “If”: Assume that the stochastic process $\Sigma$ is described by the stochastic law of the sequence $w(\cdot)$ satisfying (3). We first recall some facts concerning the topological vector space $(\mathbb{R}^{n})^{\mathbb{Z}}$ and polynomial operators in the shift, which can be found in (Willems, 1989, §4). The space of time series $(\mathbb{R}^{n})^{\mathbb{Z}}$ when equipped with the product topology is a completely metrizable and separable (i.e. Polish) topological vector space. Also, $\mathfrak{L}:=\mathrm{Ker}_{\infty}R$ is a closed and linear subspace of $(\mathbb{R}^{n})^{\mathbb{Z}}$. The polynomial operator in the shift $R(\sigma)$ is a linear, continuous, and surjective (since of full row normal rank) operator from $(\mathbb{R}^{n})^{\mathbb{Z}}$ to $(\mathbb{R}^{m})^{\mathbb{Z}}$. Consider now the quotient space $(\mathbb{R}^{n})^{\mathbb{Z}}/\mathfrak{L}$. Since $\mathfrak{L}$ is a closed and linear subspace of $(\mathbb{R}^{n})^{\mathbb{Z}}$, this is again a Polish space, as it is separable (as every quotient space of a separable space) and completely metrizable w.r.t. the induced quotient topology (see e.g. (Bourbaki, 2003, Ch.1 §3.2)). Hence by taking the restriction of $R(\sigma)$ to the quotient space $(\mathbb{R}^{n})^{\mathbb{Z}}/\mathfrak{L}$, i.e. $R|_{(\mathbb{R}^{n})^{\mathbb{Z}}/\mathfrak{L}}$, we obtain a continuous and bijective operator between Polish spaces. From this fact it follows from (Kechris, 2012, Thm. 15.1) that $R|_{(\mathbb{R}^{n})^{\mathbb{Z}}/\mathfrak{L}}$ is a Borel isomorphism, i.e. both $R|_{(\mathbb{R}^{n})^{\mathbb{Z}}/\mathfrak{L}}$ and its inverse are Borel measurable. This implies that to each event set $E_{e}\in\mathscr{B}((\mathbb{R}^{m})^{\mathbb{Z}})$ with associated probability $P_{e}(E_{e})$ corresponds one and only one $E:=R^{-1}[E_{e}]\in\mathscr{B}((\mathbb{R}^{n})^{\mathbb{Z}}/\mathfrak{L})$ with probability $P(E):=P_{e}(E_{e})$, where $R^{-1}[A]$ denotes the pre-image of $A$ under $R(\sigma)$. Hence $w(\cdot)$ defines the LTI stochastic process $(\mathbb{Z},\mathbb{R}^{n},\mathscr{B}((\mathbb{R}^{n})^{\mathbb{Z}}/\mathfrak{L}),P)$, with $\mathfrak{L}=\mathrm{Ker}_{\infty}R$ being a complete LTI behavior.

“Only if”: Assume now that $\mathfrak{L}$ is a complete LTI behavior and $\Sigma=(\mathbb{Z},\mathbb{R}^{n},\mathscr{E},P)$ an LTI process with fiber $\mathfrak{L}$. Since $\mathfrak{L}$ is complete, there exists a Laurent polynomial matrix $R(z)\in\mathbb{R}[z,z^{-1}]^{m\times n}$, $\mathrm{rk}(R)=m\leq n$, such that $\mathfrak{L}=\mathrm{Ker}_{\infty}R$ (Willems, 1989, §4). As before, $R(\sigma)$ restricted to $(\mathbb{R}^{n})^{\mathbb{Z}}/\mathfrak{L}$ is a Borel isomorphism between $(\mathbb{R}^{n})^{\mathbb{Z}}/\mathfrak{L}$, equipped with the quotient topology, and $(\mathbb{R}^{m})^{\mathbb{Z}}$, equipped with the product topology. Consider

[TABLE]

where $e(\cdot)$ describes a stochastic process with signal space $\mathbb{R}^{m}$, $\sigma$-algebra $\mathscr{B}((\mathbb{R}^{m})^{\mathbb{Z}})$, and probability $P_{e}$ such that $P_{e}(R[E]):=P(E)$ for all $E\in\mathscr{B}((\mathbb{R}^{n})^{\mathbb{Z}}/\mathfrak{L})$, being $R[E]\in\mathscr{B}((\mathbb{R}^{m})^{\mathbb{Z}})$ the image of $E$ under $R$. From this construction it follows that the LTI stochastic process $((\mathbb{R}^{n})^{\mathbb{Z}},\mathscr{B}((\mathbb{R}^{n})^{\mathbb{Z}}/\mathfrak{L}),P)$ is described by the stochastic law of $w(\cdot)$ in (3). This concludes the proof. ∎

Proof of Theorem 9.  “If”: Assume that the normal rank condition in (4) holds. Firstly, observe that this implies that $m+p\leq n$, otherwise equality in (4) can not be attained. Secondly, as noticed in the proof of Theorem 4, the operator $R_{1}$, when restricted to the domain $(\mathbb{R}^{n})^{\mathbb{Z}}/\mathrm{Ker}_{\infty}R_{1}$, is a Borel isomorphism between topological spaces $(\mathbb{R}^{n})^{\mathbb{Z}}/\mathrm{Ker}_{\infty}R_{1}$ and $(\mathbb{R}^{m})^{\mathbb{Z}}$. A similar result holds for $R_{2}$. This implies that $R_{1}$ ($R_{2}$, respectively) establishes a one-to-one correspondence between Borel sets in $(\mathbb{R}^{m})^{\mathbb{Z}}$ ($(\mathbb{R}^{p})^{\mathbb{Z}}$) and Borel sets in $(\mathbb{R}^{n})^{\mathbb{Z}}/\mathrm{Ker}_{\infty}R_{1}$ ($(\mathbb{R}^{n})^{\mathbb{Z}}/\mathrm{Ker}_{\infty}R_{2}$). Therefore every Borel set $\bar{E}_{1}\in\mathscr{B}((\mathbb{R}^{m})^{\mathbb{Z}})$ and $\bar{E}_{2}\in\mathscr{B}((\mathbb{R}^{p})^{\mathbb{Z}})$ uniquely determines events $E_{1}:=R_{1}^{-1}[\bar{E}_{1}]\in\mathscr{E}_{1}$ and $E_{2}:=R^{-1}_{2}[\bar{E}_{2}]\in\mathscr{E}_{2}$, respectively. Now, since (4) holds, we have that the polynomial operator in the shift

[TABLE]

is a linear, continuous, and surjective operator from $(\mathbb{R}^{n})^{\mathbb{Z}}$ to $(\mathbb{R}^{m+p})^{\mathbb{Z}}$ (Willems, 1989, §4). Therefore, from

1. (i)

surjectivity of $R$, and 2. (ii)

the fact that $R_{1}|_{(\mathbb{R}^{n})^{\mathbb{Z}}/\mathrm{Ker}_{\infty}R_{1}}$ and $R_{2}|_{(\mathbb{R}^{n})^{\mathbb{Z}}/\mathrm{Ker}_{\infty}R_{2}}$ are Borel isomorphisms,

it follows that, for any non-empty event $E_{1}\in\mathscr{E}_{1}$ and $E_{2}\in\mathscr{E}_{2}$, the intersection $E_{1}\cap E_{2}$ uniquely determines the set $E_{1}$ and $E_{2}$. (In particular, $E_{1}\cap E_{2}$ is non-empty, if $E_{1}$ and/or $E_{2}$ are so.) This in turn implies that $\mathscr{E}_{1}$ and $\mathscr{E}_{2}$ are complementary $\sigma$-algebras.

“Only if”: We prove the contrapositive, that is, if (4) does not hold then $\mathscr{E}_{1}$ and $\mathscr{E}_{2}$ are not complementary. Assume that (4) does not hold. Then the polynomial operator in the shift $R(\sigma):=[R_{1}(\sigma)^{\top}\ R_{2}(\sigma)^{\top}]^{\top}$ is not surjective, since the rows of $R(z)$ are linear dependent for every $z\in\mathbb{C}\setminus\{0\}$. Hence there exist two non-empty sets $E_{1}\in\mathscr{E}_{1}$ and $E_{2}\in\mathscr{E}_{2}$ whose intersection is the empty set. This in turn implies $E_{1}\cap E_{2}^{\mathrm{c}}=E_{1}=E_{1}\cap(\mathbb{R}^{n})^{\mathbb{Z}}$. Therefore it follows that $\mathscr{E}_{1}$ and $\mathscr{E}_{2}$ are not complementary.∎