Near-ideal predictors and causal filters for discrete time signals

TL;DR

This paper introduces near-ideal linear predictors and causal filters for discrete time signals with spectrum degeneracy, using polynomial approximations of ideal transfer functions to achieve effective causality.

Contribution

It proposes a novel method for designing causal filters and predictors based on polynomial approximation of non-causal transfer functions for signals with spectrum degeneracy.

Findings

Effective causal filters for spectrum-degenerate signals

Polynomial approximation of transfer functions improves predictor performance

Applicable to a range of discrete time signal processing tasks

Abstract

The paper presents linear predictors and causal filters for discrete time signals featuring some different kinds of spectrum degeneracy. These predictors and filters are based on approximation of ideal non-causal transfer functions by causal transfer functions represented by polynomials of Z-transform of the unit step signal.

Equations

$$X(z)=\sum _{t=-\infty }^{\infty }x(t)z^{-t},z\in \mathbb{T}.$$

$$x(t)=\frac{1}{2\pi }{\int }_{-\pi }^{\pi }X\left(e^{i\omega }\right)e^{i\omega t}d\omega ,t=0,\pm 1,\pm 2,....$$

$$\underset{t\in \mathbb{Z},t\le \tau -T}{\sup }|x(t+T)-{\tilde{y}}_d(t)|\to 0asd\to +\infty \forall x\in \mathcal{X},$$

$${\tilde{y}}_d={\tilde{p}}_d(x(\cdot )).$$

$$\underset{t\in \mathbb{Z},t\le \tau -T}{\sup }|x(t+T)-{\tilde{y}}_d(t)|\to 0asd\to +\infty uniformly\; inx\in \mathcal{X},$$

$$\underset{t\in \mathbb{Z}}{\sup }|x(t+T)-{\tilde{y}}_d(t)|\to 0asd\to +\infty \forall x\in \mathcal{X},$$

$$\tilde{x}={\mathcal{Z}}^{-1}({\Phi }_{\Omega }X),{\tilde{y}}_d={\tilde{p}}_d(x(\cdot )).$$

$$\underset{t\in \mathbb{Z}}{\sup }|\tilde{x}(t)-{\tilde{y}}_d(t)|\to 0asd\to +\infty uniformly\; inx\in \mathcal{X},$$

$$\psi (z)=\sum _{k=0}^d\frac{a_k}{(1-z^{-1}{)}^k},$$

$$\underset{\omega \in [-\pi ,\pi ]:|\omega |\ge \bar{\Omega }}{\sup }|\zeta (\omega )-{\psi }_d\left(e^{i\omega }\right)|\le \epsilon .$$

$$2\sum _{t=-\infty }^{-1}\cos (\omega t)x(t)+x(0)=0for\omega \in (-\bar{\Omega },\bar{\Omega }).$$

$$\sum _{t=-\infty }^{-1}\sin (\omega t)x(t)=0for\omega \in (-\bar{\Omega },\bar{\Omega }).$$

$$\underset{t\in \mathbb{Z},t\le \tau -T}{\sup }|x(t+T)-{\hat{y}}_d(t)|\le \epsilon$$

$$\underset{t\in \mathbb{Z}}{\sup }|\tilde{x}(t)-{\hat{y}}_d(t)|\le \epsilon$$

$$H_{k+1}(z)=H_1(z)H_k(z),h_{k+1}(x(\cdot ))=h_1(h_k(x(\cdot ))),k=0,1,2,3,...$$

$$H_{1,m}(z):=\mathcal{Z}{\mathbf{h}}_{1,m}=\frac{1-z^{-m}}{1-z^{-1}},R_m(z):=\mathcal{Z}({\mathbf{h}}_1-{\mathbf{h}}_{1,m})=\frac{z^{-m}}{1-z^{-1}}.$$

$${\int }_{-\pi }^{\pi }{R}_m\left(e^{i\omega }\right)e^{i\omega t}X\left(e^{i\omega }\right)d\omega ={\int }_{I_{\bar{\Omega }}}\frac{e^{-im\omega }}{1-e^{-i\omega }}{e}^{i\omega t}X\left(e^{i\omega }\right)d\omega \to 0asm\to +\infty$$

$$h_1(x(\cdot ))(t)=\sum _{s=-\infty }^{t}x(s)=\underset{m\to +\infty }{\lim }{\mathcal{Z}}^{-1}(R_m+\mathcal{Z}{\mathbf{h}}_{1,m})(t)=\underset{m\to +\infty }{\lim }\sum _{s=-m}^{t}x(s),$$

$$h_k(x(\cdot ))(t)=\sum _{s=-\infty }^t(h_{k-1}(x(\cdot ))(s),k=1,2,3,....$$

$$p_d(x(\cdot ))(t)=\sum _{k=0}^d{a}_k{h}_k(x(\cdot ))(t),$$

$$h_k(x(\cdot ))(t)=\sum _{s_{k-1}=-\infty }^t\sum _{s_{k-2}=-\infty }^{s_{k-1}}...\sum _{s_1=-\infty }^{s_2}\sum _{s=-\infty }^{s_1}x(s).$$

$${\eta }_k:=x_k(t_1-1).$$

$${\hat{y}}_d(t)=a_{0}x(t)+\sum _{k=1}^d{a}_k\left(\sum _{l=1}^k{c}_l(t){\eta }_l+f_k(t)\right).$$

$$f_k(t)=\sum _{{\tau }_1=t_1}^t\sum _{{\tau }_2=t_1}^{{\tau }_1}...\sum _{s=t_1}^{{\tau }_k}{x}_0(s),$$

$$c_1(t)=\sum _{{\tau }_1=t_1}^t\sum _{{\tau }_2=t_1}^{{\tau }_1}...\sum _{{\tau }_k=t_1}^{{\tau }_{k-1}}({\tau }_k-t_1+1),c_2(t)=\sum _{{\tau }_1=t_1}^t\sum _{{\tau }_2=t_1}^{{\tau }_1}...\sum _{{\tau }_{k-1}=t_1}^{{\tau }_{k-2}}({\tau }_{k-1}-t_1+1),$$

$$c_l(t)=\sum _{{\tau }_1=t_1}^t\sum _{{\tau }_2=t_1}^{{\tau }_1}...\sum _{{\tau }_{l-1}=t_1}^{{\tau }_{k-l}}({\tau }_{l-1}-t_1+1),l=1,2,...,k-2,$$

$$c_{k-1}(t)=\sum _{{\tau }_1=t_1}^t({\tau }_1-t_1+1),c_k(t)=t-t_1+1.$$

$$y_d(t)=y_d(t,x(t),{\bar{\eta }}_1,...,{\bar{\eta }}_d):=a_{0}x(t)+\sum _{k=1}^d{a}_k\left(\sum _{l=1}^k{c}_l(t){\bar{\eta }}_l+f_k(t)\right).$$

$$|x(t+T)-y_d(t,x(t),{\bar{\eta }}_1,...,{\bar{\eta }}_d)|\le \epsilon ,t=t_1,...,\theta -T.$$

$$a_{0}x(t_m)+\sum _{k=1}^d{a}_k\left(\sum _{l=1}^k{c}_l(t_m){\bar{\eta }}_l+f_k(t_m)\right)={\zeta }_m,m=1,...,\bar{d}.$$

Full text

††Accepted to Problems of Information Transmission

%

Key words: discrete time signals, forecasting, predictors, filters, causal transfer functions, causal approximation, high frequency signals, low frequency signals.

1 Introduction

It is well known that certain degeneracy on the spectrum can ensure opportunities for prediction and interpolation of the signals; see, e.g., [1]-[9]. The present paper considers discrete time signals in the deterministic setting, where only a single trajectory of the signal is observed, rather than a set of samples of trajectories that would allow to apply statistical methods. The method that we use is based on the frequency analysis. It is known in principle that these signals are predictable, i.e., they allow unique extrapolations from their past observations, if they have a finite spectrum gap, i.e. a segment of the unit circle ${\mathbb{T}}=\{z\in{\bf C}:\ |z|=1\}$, where their Z-transform vanish; see, e.g. [10]. This gap can be arbitrarily small and can be even reduced to a point, under certain conditions on the rate of spectrum degeneracy in a neighbourhood of this point. Respectively, an ideal low-pass filter or high-pass-filter would convert a non-predictable signal to predictable one. This is why these ideal filters cannot be causal.

For discrete time signals, some predictors based on irrational causal transfer functions were obtained in [10, 11]. The corresponding transfer functions were presented via exponentials of rational functions or power functions. In [12], some low-pass filters were also constructed based on a similar principle.

The paper addresses again the prediction and filtering problems for discrete time signals; it offers new predictors and causal filters approximating ideal filters. The causal transfer functions for these predictors and filters are represented as polynomials of Z-transform of the unit step signal, i.e., polynomials of $(1-z^{-1})^{-1}$. For the predictors, the corresponding transfer functions approximate the function $e^{i\omega T}$ on ${\mathbb{T}}$, where $\omega\in(-\pi,\pi]$ represents the frequency, and where an integer $T>0$ represents a preselected prediction horizon. For the filters, the corresponding transfer functions approximate the real valued step function representing the trace on ${\mathbb{T}}$ of Z-transform of an ideal filter. The approximation is possible for signals with some arbitrarily small spectrum gap; the resulting signal could have a wider preselected spectrum gap. This polynomial approximation method is based on the approach developed in [13, 14] for prediction of continuous time signals.

The results are applicable for high frequency signals as well as for signal for a spectrum gap located anywhere on ${\mathbb{T}}$, for example, low frequency signals. Moreover, the paper shows that some signals with a non-degenerate spectrum also can be predicted in a half of the timeline given some conditions on some spectrum type characteristics of the trace on this half of timeline.

These new predictors and filters allow an explicit representation in the time domain and in the frequency domain; in addition, they are independent on the spectral characteristics of the input signals with given type of the spectrum degenerocity. Some computational approach based on model fitting is suggested.

The paper is organized in the following manner. In Section 2, we formulate the definitions. In Section 3, we formulate the main theorems on predictability and predictors (Theorem 1 and Theorem 2). In Section 4, we discuss representation of transfer functions in the time domain. In Section 5, we discuss some implementation problems. In Section 6, a method of computing approximating functions for exponentials $e^{i\omega}$ is suggested. In Section 7, we suggest extension of the results on low frequency and other signals. Section 8 contains the proofs.

2 Problem setting

Some notations

Let ${\mathbb{Z}}$ be the set of all integers.

We denote by $\ell_{r}$ the set of all functions (signals) $x:{\mathbb{Z}}\to{\bf C}$, such that $\|x\|_{\ell_{r}}:=\left(\sum_{t=-\infty}^{\infty}|x(t)|^{r}\right)^{1/r}<+\infty$ for $r\in[1,\infty)$.

For $x\in\ell_{1}$ or $x\in\ell_{2}$, we denote by $X={\cal Z}x$ the Z-transform

[TABLE]

Respectively, the inverse Z-transform $x={\cal Z}^{-1}X$ is defined as

[TABLE]

If $x\in\ell_{2}$, then $X|_{\mathbb{T}}$ is defined as an element of $L_{2}({\mathbb{T}};{\bf C})$.

We denote by ${\mathbb{I}}$ the indicator function.

Some definitions

Let either $E={\bf R}$ or $E={\bf C}$.

Let ${\cal X}\subset\ell_{\infty}$ be a set currently observable discrete time signals with values in $E$.

Let ${\cal P}({\cal X})$ be the set of all continuous mappings $p:{\cal X}\to\ell_{\infty}$ such that, for any $x_{1},x_{2}\in{\cal X}$ and $\bar{t}\in{\mathbb{Z}}$, we have that $p(x_{1}(\cdot))(t)=p(x_{2}(\cdot))(t)$ for all $t\leq\bar{t}$ if $x_{1}(t)=x_{2}(t)$ for all $t\leq\bar{t}$. In other words, this is the set of ”causal” mappings; we will look for predictors and filters in this class.

Let us consider first a prediction problem. Let an integer $T\geq 1$ be given. The goal is to estimate, at current times $t$, the values $x(t+T)$, using historical values of the observable process $x(s)|_{s\leq t}$. Therefore, $T$ is the prediction horizon in this setting.

Definition 1

Let ${\cal X}\subset\ell_{\infty}$ and $\tau\in\{-1,0,+\infty\}$.

• (i)

We say that the class ${\cal X}$ is predictable with the prediction horizon $T$ up to time $\tau$ if there exists a sequence $\{\widetilde{p}_{d}(\cdot)\}_{d=1}^{+\infty}\subset{\cal P}({\cal X})$ such that

[TABLE]

where

[TABLE]

• (ii)

We say that the class ${\cal X}$ is uniformly predictable with the prediction horizon $T$ up to time $\tau$ if there exists a sequence $\{\widetilde{p}_{d}(\cdot)\}_{d=1}^{+\infty}\subset{\cal P}$ such that

[TABLE]

where $\widetilde{y}_{d}(\cdot)$ is as in part (i) above.

Functions $\widetilde{y}_{d}(t)$ in the definition above can be considered as approximate predictions of the process $x(t+T)$.

Let us consider now the filtering problem.

Let $\Omega\in(0,\pi)$ be given. Let a function $\Phi_{\Omega}:{\mathbb{T}}\to{\bf R}$ be defined such that $\Phi_{\Omega}\left(e^{i\omega}\right)={\mathbb{I}}_{|\omega|\geq\Omega}$.

We consider an ideal high-pass filter such that the trace of its transfer function on ${\mathbb{T}}$ is $\Phi_{\Omega}\left(e^{i\omega}\right)$, $\omega\in(-\pi,\pi]$, i.e., filters with the the suppression interval $(-\Omega,\Omega)$.

The goal is to find an arbitrarily close approximation of this non-causal transfer function $\Phi_{\Omega}\left(e^{i\omega}\right)$ by causal transfer functions.

Definition 2

Let ${\cal X}\subset\ell_{\infty}$.

• (i)

We say that a class ${\cal X}\subset\ell_{2}$ allows causal high-pass filtering with the suppression interval $(-\Omega,\Omega)$ if there is a sequence $\{\widetilde{p}_{d}(\cdot)\}_{d=1}^{+\infty}\subset{\cal P}(\bar{\cal X})$ such that

[TABLE]

where

[TABLE]

• (ii)

We say that the class ${\cal X}$ allows uniform causal high-pass filtering with the suppression interval $(-\Omega,\Omega)$ if there exists a sequence $\{\widetilde{p}_{d}(\cdot)\}_{d=1}^{+\infty}\subset{\cal P}$ such that

[TABLE]

where $\widetilde{x}$ and $\widetilde{y}_{d}$ are as in part (i) above.

In the last definition, operators $p_{d}$ represent causal near-ideal high pass filters; they ensure, for the class ${\cal X}$, an arbitrarily close approximation of the non-causal ideal high-pass filter defined by its transfer function $\Phi_{\Omega}$.

3 The main result

For $d=0,1,2,...$, let $\Psi_{d}^{E}$ be the set of all functions $\psi:{\bf C}\setminus\{1\}\to{\bf C}$ represented as

[TABLE]

where $a_{k}\in E$ can be any. Let $\Psi^{E}:=\cup_{d}\Psi_{d}^{E}$.

Lemma 1

For $\bar{\Omega}\in(0,\pi)$, let the function $\zeta:[-\pi,\pi]\to{\bf C}$ be defined either as $\zeta(\omega)=e^{i\omega T}$ or as $\zeta(\omega)={\mathbb{I}}_{|\omega|\geq\bar{\Omega}}$. Then, for any $\varepsilon>0$, there exists a integer $d=d(\varepsilon,T)>0$ and $\psi_{d}\in\Psi_{d}^{\bf R}$ such that

[TABLE]

For $\bar{\Omega}\in(0,\pi)$, let ${\cal X}(\bar{\Omega})$ be the set of all signals $x:{\mathbb{Z}}\to E$ such that $x(\cdot)\in\ell_{2}$ and $X\left(e^{i\omega}\right)=0$ for $\omega\in(-\bar{\Omega},\bar{\Omega})$ and $X={\cal Z}x$.

Further, for $\tau=-1,0$, let ${\cal X}(\tau,\bar{\Omega})$ be the set of all real signals $x:{\mathbb{Z}}\to{\bf R}$ such that $x(\cdot)\in\ell_{2}$ and the following holds:

• •

If $\tau=0$, then

[TABLE]

• •

If $\tau=-1$, then

[TABLE]

We say that the processes from ${\cal X}(\tau,\bar{\Omega})$ described above feature will call a left-sided spectrum degeneracy.

The feature of the processes from ${\cal X}(\tau,\bar{\Omega})$, $\tau=-1,0$, described above, we will call a left-sided spectrum degeneracy. In addition, we define ${\cal X}(\infty,\bar{\Omega})$ as ${\cal X}(\bar{\Omega})$.

Theorem 1

Let $\Omega\in(0,\pi)$ be given, $\tau\in\{-1,0,+\infty\}$. The predictability up to time $\tau$ for $x\in{\cal X}(\tau,\Omega)$ considered in Definition 1(i), as well as the uniform predictability up to time $\tau$ for $x\in{\cal X}(\tau,\Omega)\cap\{x\in\ell_{2}:\ \|x\|_{\ell_{2}}\leq 1\}$ considered in Definition 1(ii), can be ensured with the sequence of the predictors $p_{d}:{\cal X}(\Omega)\to\ell_{2}$, $d=1,2,....,$ defined by their transfer functions $\psi_{d}(z)$ selected as in Lemma 1 with $\zeta(\omega)=e^{i\omega T}$. More precisely, for any $\bar{\varepsilon}>0$ and $\widehat{y}_{d}(t)=p_{d}(x(\cdot))(t)$, the estimate

[TABLE]

holds if $d$ and $\psi_{d}$ are such that (2) holds with $\zeta(\omega)=e^{i\omega T}$ for sufficiently small $\varepsilon$.

Theorem 2

For $\Omega\in(0,\pi)$ and any $\Omega_{0}\in(0,\Omega)$, for $x\in{\cal X}(\Omega_{0})$, the causal filtering considered in Definition 2(i), as well as the uniform causal filtering for $x\in{\cal X}(\Omega_{0})\cap\{x\in\ell_{2}:\ \|x\|_{\ell_{2}}\leq 1\}$ considered in Definition2(ii) can be ensured with the sequence of the causal filters $p_{d}:{\cal X}(\Omega)\to\ell_{2}$, $d=1,2,....,$ defined by their transfer functions $\psi_{d}(z)$ selected as in Lemma 1 with $\zeta(\omega)={\mathbb{I}}_{|\omega|\geq\Omega}$. More precisely, for any $\bar{\varepsilon}>0$ and $\widehat{y}_{d}(t)=p_{d}(x(\cdot))(t)$ and $\widetilde{x}={\cal Z}^{-1}(\Phi_{\Omega}X)$, the estimate

[TABLE]

if $d$ and $\psi_{d}$ are such that (2) holds with $\zeta(\omega)={\mathbb{I}}_{|\omega|\geq\Omega}$ for sufficiently small $\varepsilon$.

According to this theorem, a process with an arbitrarily small spectrum gap $(-\Omega_{0},\Omega_{0})$ can be converted, using causal operations, into a process with larger spectrum gap up to $(-\Omega,\Omega)$.

It van be noted that:

• •

The transfer functions $\psi_{d}(z)$ are analytic in the domain ${\bf C}\setminus\{1\}$. If we apply their traces $\psi_{d}\left(e^{i\omega}\right)|_{\omega\in(-\pi,\pi]}$ on ${\mathbb{T}}$ for calculation of the outputs for inputs $x\in{\cal X}(\Omega)$, then we obtain the same outputs as for the functions $\psi_{d}\left(e^{i\omega}\right){\mathbb{I}}_{\omega\in(-\pi,\pi],|\omega|\geq\Omega}$.

• •

For real valued inputs $x$, the outputs of these predictors and filters are real valued.

• •

$p_{d}(\cdot)$ depends on $T$ and $\Omega$ via the coefficients $a_{k}$ in the setting of Theorem 1, and $p_{d}(\cdot)$ depends on $\Omega$ and $\Omega_{0}$ via the coefficients $a_{k}$ in the setting of Theorem 2.

4 Representation of operators $p_{d}(\cdot)$ in the time domain

Let either $\bar{\Omega}=\Omega$ or $\bar{\Omega}=\Omega_{0}$.

Consider operators $h_{k}$ defined on ${\cal X}(\bar{\Omega})$ by their transfer functions $H_{k}(z)=(1-z^{-1})^{-k}$, $k=0,1,2,...$ . In other words, if $y=h_{k}(x)$ for $x\in{\cal X}(\bar{\Omega})$, then $Y(z)=(1-z^{-1})^{-k}X(z)$ for $Y={\cal Z}y$ and $X={\cal Z}x$. Clearly,

[TABLE]

Hence $h_{k}(x(\cdot))\in{\cal X}(\bar{\Omega})$ for all $k=0,1,2,...$, $x\in{\cal X}(\bar{\Omega})$. Therefore, the Z-transforms of processes $h_{k}(x(\cdot))$ vanish on $\{e^{i\omega},\ \omega\in[-\pi,\pi],\ |\omega|<\bar{\Omega}\}$, and the operators $h_{k}:{\cal X}(\bar{\Omega})\to\ell_{2}$ are continuous, assuming that ${\cal X}(\bar{\Omega})$ is a subspace of $\ell_{2}$ provided with $\ell_{2}$-norm.

Let ${\bf h}_{1}\in\ell_{\infty}$ be defined such that ${\bf h}_{1}(t)=0$ for $t<0$ and ${\bf h}_{1}(t)=1$ for $t\geq 0$, i.e. ${\bf h}_{1}={\cal Z}^{-1}H_{1}(z)$.

Let $I_{\bar{\Omega}}:=[-\pi,-\bar{\Omega}]\cup[\bar{\Omega},\pi]$ and $x\in{\cal X}(\bar{\Omega})$.

Let us show that, in the time domain, the operator $h_{1}(x(\cdot))$ can be represented via causal convolution with the kernel ${\bf h}_{1}$, i.e. if $x\in{\cal X}(\bar{\Omega})$ then $h_{1}(x(\cdot))(t)=\sum_{s=-\infty}^{t}x(s)$.

Let ${\bf h}_{1,m}(t)={\bf h}_{1}(t){\mathbb{I}}_{\{t<m-1\}}$. Clearly, ${\bf h}_{1,m}\in\ell_{2}$. Let

[TABLE]

Clearly, $(1-e^{-i\cdot})^{-1}e^{i\cdot}X(e^{i\cdot})\in L_{2}(I_{\bar{\Omega}},{\bf C})$ for any $t$. Hence

[TABLE]

for each $t\in{\mathbb{Z}}$. It follows that if $x\in{\cal X}(\bar{\Omega})$ then

[TABLE]

and the series converges for each $t\in{\mathbb{Z}}$.

This implies that

[TABLE]

Therefore, the operators $p_{d}$ in Theorems 1-2 can be represented as

[TABLE]

where

[TABLE]

All series here converge as described above for $h_{1}$.

It can be noted that $x\in{\cal X}(\bar{\Omega})\cap\ell_{1}$ then the series $\sum^{t}_{s=-\infty}x(s)$ converges absolutely; however, for general type $x\in{\cal X}(\bar{\Omega})$, there is no guarantee that $x\in\ell_{1}$ or $h_{k}(x(\cdot))\in\ell_{1}$.

5 On numerical implementation of Theorems 1-2

The direct implementation of the predictors introduced in Theorems 1-2 requires evaluation of sums for semi-infinite series that is not practically feasible. However, these theorems could lead to predicting methods bypassing this calculation. Let us discuss these possibilities.

Let $t_{1}\in{\bf R}$ be given such that $t_{1}<\tau$, where $\tau$ in the setting of Theorem 1 is such as described therein, and $\tau=+\infty$ in the setting of Theorem 2. Let $x_{k}:=h_{k}(x)$ for $x\in{\cal X}(\Omega_{0})$, $k=1,2,...$, and let

[TABLE]

Lemma 2

In the notation of Theorems 1-2, for any $t$ such that $t_{1}\leq t<\tau+1$, we have that $\widehat{y}_{d}=p_{d}(x(\cdot))$ can be represented as

[TABLE]

Here $a_{k}\in{\bf R}$ are the coefficients for $\psi_{d}(z)=\sum_{k=1}^{d}a_{k}(1-z^{-1})^{-k}$ from Theorems 1-2,

[TABLE]

[TABLE]

and

[TABLE]

[TABLE]

This lemma shows that calculation of $\widehat{y}_{d}(t)=p_{d}(x(\cdot))(t)$ is easy for $t>t_{1}$ if we know all $\eta_{k}$, $k=1,...,d$, and observe $x(s)|_{s=t_{1},...,t}$.

Let us discuss some ways to evaluate $\eta_{k}$ bypassing summation of infinite series.

First, let us observe that (6) implies a useful property given below.

Corollary 1

For any $\varepsilon>0$, there exist an integer $d=d(\varepsilon)>0$ and $a_{0},a_{1},....,a_{d}\in{\bf R}$ such that, for any $t_{1}\in{\mathbb{Z}}$, there exist $\bar{\eta}_{1},\bar{\eta}_{1},...,\bar{\eta}_{d}\in{\bf R}$ such that $|\widetilde{x}(t)-y_{d}(t)|\leq\varepsilon$ for all $t\geq t_{1}$, where

[TABLE]

In this corollary, $d=d(\varepsilon)$ and $a_{k}\in{\bf R}$ are such as defined in Theorems 1-2.

The case of prediction problem: Theorem 1 setting

Let us discuss using (6) and (7) for evaluation of $\eta_{k}$ in Theorem 1 setting.

Let $\theta>t_{1}$ and $\theta<\tau+1$. Assume first that the goal is to forecast the value $\widetilde{x}(t)=x(t+T)$ given observations at times $t\leq\theta$, in the setting of Theorems 1. It appears that if $\theta>t_{1}+T$ then Corollary 1 gives an opportunity to construct predictors via fitting parameters $\eta_{0},...,\eta_{d}$ using past observations available for $t=t_{1},...,\theta-T$: we can match the values $y_{d}(t,x(t),\bar{\eta}_{1},...,\bar{\eta}_{d})$ with the past observations $x(t+T)$. Starting from now, we assume that $\theta>t_{1}+T$.

Let $d$ be large enough such that $x(t+T)$ is approximated by $\widehat{y}_{d}(t)$ as described in Theorem 1, i.e., $\sup_{t\in{\mathbb{Z}}}|x(t+T)-\widehat{y}_{d}(t)|\leq\varepsilon$ for some sufficiently small $\varepsilon>0$, for some choice of $a_{k}$.

As an approximation of the true $\eta_{1},...,\eta_{d}$, we can accept a set $\bar{\eta}_{1},...,\bar{\eta}_{d}$ such that

[TABLE]

(Remind that, at time $\theta$, values $x(t+T)$ and $y_{d}(t,x(t),\bar{\eta}_{1},...,\bar{\eta}_{d})$ are observable for these $t=t_{1},....,\theta-T$). If (8) holds, we can conclude that $y_{d}(t,x(t),\bar{\eta}_{1},...,\bar{\eta}_{d})$ delivers an acceptable prediction of $x(t+T)$ for these $t$. Clearly, Theorem 1 implies that a set $\bar{\eta}_{1},...,\bar{\eta}_{d}$ ensuring (8) exists since this inequality holds with $\bar{\eta}_{k}=\eta_{k}$.

The corresponding value $y_{d}(\theta,x(t),\bar{\eta}_{1},...,\bar{\eta}_{d})$ would give an estimate for $\widehat{y}_{d}(\theta)$ and, respectively, for $x(\theta+T)$.

Furthermore, finding a set $\bar{\eta}_{1},...,\bar{\eta}_{d}$ that ensures (8) could still be difficult. Instead, one can consider fitting predictions and observations at a finite number of points $t=t_{1},...,T-\theta$.

Let a integer $\bar{d}\geq d$ and a set $\{t_{m}\}_{m=1}^{\bar{d}}\subset{\mathbb{Z}}$ be selected such that $t_{1}<t_{2}<t_{3}<...<t_{\bar{d}-1}<t_{\bar{d}}\leq\theta-T$. We suggest to use observations $x(t)$ at times $t=t_{m}$. Consider a system of equations

[TABLE]

Consider first the case where $\bar{d}=d$. In this case, we can select $\zeta_{m}=x(t_{m}+T)$; these values are directly observable, without calculation of semi-infinite series required for $\widehat{y}_{d}(t_{m})$. The corresponding choice of $\bar{\eta}_{k}$ ensures zero prediction error for $x(t_{m}+T)$, $m=1,...,\bar{d}$.

Including into consideration more observations, i.e., selecting larger $\bar{d}>d$ and larger set $\{t_{1},....,\theta-T\}$, would improve estimation of $\eta_{k}$. If we consider $\bar{d}>d$, then, in the general case, it would not be feasible to achieve that $y_{d}(t,\bar{\eta}_{1},...,\bar{\eta}_{d})=x(t_{m}+T)$ for all $m$, since it cannot be guaranteed that system (9) is solvable for $\zeta_{m}\equiv x(t_{m}+T)$: the system will be overdefined. Nevertheless, estimate presented in (8) can still be achieved for any arbitrarily large $\bar{d}$, since (8) holds. A solution could be found using methods for fitting linear models.

So far, the consistency of these procedures is unclear since a choice of smaller $\varepsilon$ leads to larger $d$. We leave analysis of these methods for the future research.

The case of causal filtering problem: Theorem 2 setting

In the setting of Theorem 2, the past values of the true unknown process $\widetilde{x}(t)$ are not observable and hence cannot be used for fitting the values $\eta_{1},...,\eta_{d}$. However, we can use that the values $\eta_{1},...,\eta_{k}$ in (6)-(7) are still the same as in the setting of Theorem 2, where $\widetilde{x}(t)=x(t+T)$. Since past $x(s)|_{s=t_{1},...,t}$ are observable, we can use the fitting procedure based on Theorem 1 to estimate $\eta_{1},...,\eta_{d}$ using (6)-(7) with the coefficients $a_{k}$ defined for approximation of $\zeta(\omega)=e^{i\omega T}$ and with observations $x(t_{m})$, $t_{m}\leq t$, as described above. After that, we can estimate $\widetilde{x}(t)$ using equation (6) again with the new coefficients $a_{k}$ defined for approximation of $\zeta(\omega)={\mathbb{I}}_{|\omega|\leq\Omega}$.

6 A possible choice of $\psi_{d}$ for predictors in Theorem 1 setting

The coefficients $a_{k}$ for functions $\psi_{n}$ could be found use numerical methods from classical analysis such as the Gram-Schmidt procedure. In the case of Theorem 1 for predictors, finding these coefficients can be simplified, especially for $T=1$.

Let us demonstrate this.

Assume that $T=1$. For real $\nu>0$, define on ${\bf C}\setminus\{1\}$ a function

[TABLE]

This function is a modification of the transfer function introduced in [10] for prediction of signals with a single point spectrum degeneracy. Clearly,

[TABLE]

uniformly on the set $\{e^{i\omega},\quad\omega\in(-\pi,\pi],\ |\omega|\geq\Omega\}$. Hence

[TABLE]

uniformly on the set $\{e^{i\omega},\quad\omega\in(-\pi,\pi],\ |\omega|\geq\Omega\}$.

Further, for $\varepsilon>0$, let $\nu<0$ be selected such that

[TABLE]

The function $\widetilde{\psi}(\nu,\cdot)$ is analytic in ${\bf C}\setminus\{1\}$, and is bounded on ${\bf C}\setminus\{z\in{\bf C}:\ |1-z|>\delta\}$ for any $\delta>0$. Clearly, we have that

[TABLE]

where

[TABLE]

and where convergence is uniform on the set $\{e^{i\omega},\quad\omega\in(-\pi,\pi],\ |\omega|\geq\Omega\}$.

It can be observed that the functions $\psi_{d}(\nu,z)$ belong to $\Psi_{d}$, since

[TABLE]

For example,

[TABLE]

Clearly, we can select $d$ such that

[TABLE]

For this $d$ and $\nu$, we have that

[TABLE]

The coefficients $a_{k}$ can be computed form the representation of $\psi_{d}$ as an element of $\Psi_{d}$.

For the case of $T>1$, one can use functions $\psi_{d}(\nu,z)^{T}$.

7 Low frequency and other signals

Let us show that the results obtained above for high frequency signals can be applied to signals of more general type described as follows.

Let $\Omega\in(0,\pi)$, $\Omega_{0}\in(0,\Omega)$, and $\theta\in(-\pi,\pi]$ be given, and let ${\cal Y}(\Omega,\theta)$ be the set of all signals $x\in\ell_{2}$ such that $X\left(e^{i(\omega-\theta)}\right)=0$ for $|\omega|<\Omega$, $\omega\in[-\pi,\pi]$ and $X={\cal Z}x$.

For example, ${\cal Y}(\Omega,0)={\cal X}(\Omega)$; this set includes high frequency signals such that $X\left(e^{i\omega}\right)=0$ if $|\omega|<\Omega$. Respectively, the set ${\cal Y}(\Omega,\pi)$ includes low frequency signals (band limited signals) such that $X\left(e^{i\omega}\right)=0$ if $\omega\in(-\pi,-\pi+\Omega)\cup(\pi-\Omega,\pi]$.

To predict a signal $\widehat{x}\in{\cal Y}(\Omega,\theta)$, one can convert it into a signal $x\in{\cal X}(\Omega)={\cal Y}(\Omega,0)$ as $x(t)=e^{-i\theta t}\widehat{x}(t)$. Then one can use for $x$ the predictors introduced in Theorem 1. The implied prediction $\widehat{y}(t)$ for $\widehat{x}(t)$ can be obtained as $\widehat{y}(t)=e^{i\theta t}y(t)$, where $y(t)$ is the corresponding prediction for $x(t)$.

Similarly, one can construct a causal filter that, for $x\in{\cal Y}(\Omega_{0},\theta)$, produces an approximation of $\widehat{x}\in{\cal Y}(\Omega,\theta)$ such that $\widehat{x}={\cal Z}^{-1}\Phi_{\Omega,\theta}X$, where $X={\cal Z}x$, and $\Phi_{\Omega,\theta}$ is Z-transform of an ideal filter such that $\Phi_{\Omega,\theta}\left(e^{i(\omega-\theta)}\right)={\mathbb{I}}_{|\omega|>\Omega,\ \omega\in[-\pi,\pi]}$. Again, one can convert it into a signal $x\in{\cal X}(\Omega_{0})={\cal Y}(\Omega_{0},0)$ as $x(t)=e^{-i\theta t}\widehat{x}(t)$. Then one can use for $x$ the causal filter introduced in Theorem 2. The implied filtered signal $\widehat{y}(t)$ for $\widehat{x}(t)$ can be obtained as $\widehat{y}(t)=e^{i\theta t}y(t)$, where $y(t)$ is the corresponding filtered signal for $x(t)$.

Alternatively, we can construct predictors and filters directly for signals from ${\cal Y}(\Omega,\theta)$ similarly to the ones introduced in Theorems 1-2 and with the transfer functions

[TABLE]

approximating $e^{i\omega T}$ and ${\mathbb{I}}_{|\omega|>\Omega}$ on ${\mathbb{T}}$.

In the setting where $x\in\cup_{\theta}{\cal Y}(\Omega,\theta)$, and where $\theta$ is unknown, we can use approach from Section 5 to fit $\theta$ from past observations as a new unknown parameter.

8 Proofs

Proof of Lemma 1. For a set $I\subset[-\pi,0)\cup(0,\pi]$, let $\gamma^{E}(I)$ (or $\gamma_{d}^{E}(I)$) be the set of functions $\gamma:I\to{\bf C}$ constructed as $\gamma(\omega)=\psi\left(e^{i\omega}\right)$ for some $\psi$ from $\Psi^{E}$ (or from $\Gamma_{d}^{E}(I)$, respectively)).

Let $I_{\bar{\Omega}}:=[-\pi,-\bar{\Omega}]\cup[\bar{\Omega},\pi]$.

Clearly, $\frac{1}{1-z^{-1}}=1-\frac{1}{1-z}$ for all $z\in{\bf C}$, $z\neq 1$. Hence

[TABLE]

It follows that, if $\psi(z)\in\Psi^{E}$ then $\psi(z^{-1})\in\Psi^{E}$, for both $E={\bf R}$ and $E={\bf C}$. This implies that $\overline{\gamma(\omega)}\in\Gamma^{\bf E}(I_{\bar{\Omega}})$ if $\gamma(\omega)\in\Gamma^{E}(I_{\bar{\Omega}})$.

Since the function $\gamma_{1}(\omega)={\rm Re\,}\frac{1}{1-e^{-i\omega}}$ is strictly monotone on the intervals $(-\infty,0)$ and $(0,\infty)$, and has different signs on these two intervals, it follows that $\gamma_{1}(\alpha)\neq\gamma_{1}(\beta)$ for all $\alpha,\beta\in I_{\bar{\Omega}}$, $\alpha\neq\beta$. It follows that the set of function $\Gamma^{\bf C}(I_{\Omega})$ separates points on the compact set $I_{\bar{\Omega}}$. By the Stone-Weierstrass Theorem for complex valued continuous functions on compact sets of real numbers, it follows that the set $\Gamma(I_{\bar{\Omega}})$ is complete in the space $C(I_{\bar{\Omega}};{\bf C})$ of continuous complex-valued functions defined on $I_{\Omega}$ with the supremum norm; see, e.g., Theorem 10 in [15], pp. 238. It follows that, for any $\varepsilon>0$, there exists $d>0$ and $\widehat{\gamma}_{d}\in\Gamma_{d}^{\bf C}(I_{\bar{\Omega}})$ represented as $\widehat{\gamma}_{d}(\omega)=\sum_{k=0}^{d}\frac{A_{k}}{\left(1-e^{-i\omega}\right)^{k}}$ defined for $\omega\in{\bf R}\setminus\{0\}$, where $A_{k}\in{\bf C}$, such that

[TABLE]

For $\zeta(\omega)=e^{i\omega T}$ this follows directly from Theorem 10 in [15], pp. 238, mentioned above. For $\zeta(\omega)={\mathbb{I}}_{|\omega|\geq\bar{\Omega}}$ this follows from the fact that the set $C(I_{\Omega};{\bf C})$ is everywhere dense in $L_{2}(I_{\bar{\Omega}};{\bf C})$, and convergence in $C(I_{\bar{\Omega}};{\bf C})$ implies convergence in $L_{2}(I_{\bar{\Omega}};{\bf C})$.

Let us show that the same estimate holds for $\gamma_{d}\in\Gamma_{d}^{\bf R}(I_{\Omega})$ defined as $\gamma_{d}(\omega)=\sum_{k=0}^{d}\frac{a_{k}}{\left(1-e^{-i\omega}\right)^{k}}$, where $a_{k}={\rm Re\,}A_{k}$.

Suppose that ${\rm Im\,}A_{k}\neq 0$ for some $k$. Clearly, the real and the the imaginary part of $\frac{i{\rm Im\,}A_{k}}{\left(1-e^{-i\omega}\right)^{k}}$ are even and odd, respectively. On the other hand, the functions ${\rm Re\,}e^{i\omega T}=\cos(\omega T)$ and ${\rm Im\,}e^{i\omega T}=\sin(\omega T)$ are odd and even, respectively, on ${\bf R}$. Therefore, the replacement of $A_{k}$ by $a_{k}={\rm Re\,}A_{k}$ cannot spoil the estimate. Hence the transfer function $\psi_{d}\left(e^{i\omega}\right)=\gamma_{d}(\omega)=\sum_{k=0}^{d}\frac{a_{k}}{\left(1-e^{-i\omega}\right)^{k}}$ satisfies the required estimate. This completes the prove of Lemma 1. $\Box$

Proof of Theorems 1-2. Let us consider first the case where $x\in{\cal X}(\infty,\Omega)={\cal X}(\Omega)$.

We continue with the proof for Theorems 1 (with $\tau=+\infty)$ and Theorem 2 simultaneously. For the proof of Theorem 1, we assume that $\bar{\Omega}=\Omega$ and $\zeta:I_{\Omega}\to{\bf C}$ is defined as $\zeta(\omega)=e^{i\omega T}$. For the proof of Theorem 2, we assume that $\bar{\Omega}=\Omega_{0}$ as $\zeta(\omega)={\mathbb{I}}_{|\omega|\geq\Omega}$.

Assume that estimate (2) holds for selected $d,\gamma_{d},\psi_{d}$. We have that

[TABLE]

where $\widetilde{x}(t)=x(t+T)$ in the setting of Theorem 1, and $\widetilde{x}(t)$ is an ideal filtered process in the setting of Theorem 2. Clearly,

[TABLE]

We have that

[TABLE]

Hence $|\widetilde{x}(t)-\widehat{y}_{d}(t)|\leq\varepsilon\|x\|_{\ell_{2}}$. This implies the proofs of Theorems 1 for the case where $\tau=+\infty$ and Theorem 2.

Let us prove Theorem 1 for the case where $\tau=0$. Let $x\in{\cal X}(0,\Omega)$. Let us define an even function $\widetilde{x}:{\mathbb{Z}}\to{\bf R}$ such that $\widetilde{x}(t)=x(t)$ for $t\leq 0$, and $\widetilde{x}(t)=x(-t)$ for $t>0$. Let $\widetilde{X}={\cal Z}\widetilde{x}$. It can be shown that ${\rm Re\,}\widetilde{X}\left(e^{i\omega}\right)=2\sum_{t=-\infty}^{-1}\cos(\omega t)x(t)+x(0)$ and ${\rm Im\,}\widetilde{X}\left(e^{i\omega}\right)=0$ for $\omega\in(-\pi,\pi]$. This implies that $\widetilde{x}\in{\cal X}(\Omega)$. Furthermore, since predictors $p_{d}$ are causal, it follows that $p_{d}(\widetilde{x}(\cdot))(t)=p_{d}(x(\cdot))(t)$ for all $t\leq 0$. Then the proof for $\tau=0$ follows from the proof the case of $\tau=+\infty$.

The case where $\tau=-1$ can be considered similarly. For $x\in{\cal X}(-1,\Omega)$, we define an odd function $\widetilde{x}:{\mathbb{Z}}\to{\bf R}$ such that $\widetilde{x}(t)=x(t)$ for $t\leq-1$, $\widetilde{x}(0)=0$, and $\widetilde{x}(t)=-x(-t)$ for $t>0$. Let $\widetilde{X}={\cal Z}\widetilde{x}$. It can be shown that ${\rm Re\,}\widetilde{X}\left(e^{i\omega}\right)=0$ and ${\rm Im\,}\widetilde{X}\left(e^{i\omega}\right)=2\sum_{t=-\infty}^{-1}\sin(\omega t)\widetilde{x}(t)$ for $\omega\in(-\pi,\pi]$. It follows that $\widetilde{x}\in{\cal X}(\Omega)$. Again, since predictors the $p_{d}$ are causal, it follows that $p_{d}(\widetilde{x}(\cdot))(t)=p_{d}(x(\cdot))(t)$ for all $t\leq-1$. Hence the proof for $\tau=-1$ follows from the proof for the case of $\tau=+\infty$. This completes the proofs of Theorems 1. $\Box$

Proof of Lemma 2. We have that

[TABLE]

Futrher, we have that $y_{d}(t)=a_{0}x(t)+\sum_{k=1}^{d}a_{k}x_{k}(t)$ for any $t\geq t_{1}$, i.e.,

[TABLE]

Here we assume that $x_{0}:=x$.

Furthermore, we have that

[TABLE]

and

[TABLE]

Similarly,

[TABLE]

Similarly, we obtain that, for $k>2$,

[TABLE]

It follows that

[TABLE]

Together with (10), this proves (7) and completes the proof of Lemma 2. $\Box$

9 Concluding remarks

1. i.

The approach suggested in this paper allows many modifications. In particular, other non-causal discrete time transfer functions can be approximated by causal transfer functions from $\Psi^{E}$. In fact, any transfer function $H(z)$ can be approximated that way if $\int_{-\pi}^{\pi}|H\left(e^{i\omega}\right)|^{2}d\omega<+\infty$. 2. ii.

It can be shown that, by Theorem 10 in [15], pp. 238 again, approximation of $\zeta(\omega)={\mathbb{I}}_{|\omega|\geq\bar{\Omega}}$ in Lemma 1 can be in fact achieved on the set of real valued functions represented as

[TABLE]

with $b_{k}\in{\bf R}$. This may help to streamline calculations since this set is smaller than $\Psi^{R}$. If $b_{k}$ are found, then we can derive the coefficients $a_{k}$ needed for the fitting of $\eta_{k}$ via (6)-(7). 3. iii.

The predictors introduced in [10, 11] do not allow the fitting procedure described in Section 5 since the kernels of the corresponding causal convolutions are heavily time dependent. 4. iv.

In the present paper, we consider $L_{2}$-approximation of non-causal transfer functions; this allowed to approximate discontinuous on ${\mathbb{T}}$ transfer functions used for the filtering problem. In addition, this would allow to use the Gram-Schmidt procedure to construct the functions $\psi_{d}$. This was not feasible in the continuous time setting [14], where the uniform approximation on the infinite intervals was required. 5. v.

In general, it can be expected that the approximating functions $\psi_{d}\left(e^{i\omega}\right)$ take large values for large $d$ inside the interval $(-\bar{\Omega},\bar{\Omega})$, in the terms of Lemma 1. However, some robustness of the prediction and filtering with respect to noise contamination can be established similarly to [10]. We leave it for the future research. 6. vi.

The processes from ${\cal X}(\tau,\Omega)$ do not necessarily have a spectrum degeneracy for $\tau=-1,0$ and $\Omega\in(0,\pi)$; in fact, their Z-transforms can be separated from zero on ${\mathbb{T}}$. However, Theorem 1 shows that they are predictable on the left half of the timeline because of their left-sided spectrum degeneracy defined by (3),(4).