Perturbations and projections of Kalman-Bucy semigroups

TL;DR

This paper investigates how perturbations and projections affect Kalman-Bucy semigroups and Riccati equations, providing error characterizations and introducing new models relevant to filtering, control, and stochastic processes.

Contribution

It offers new insights into the effects of perturbations and projections on Kalman-Bucy and Riccati equations, including error analysis and novel projection models like Bose-Mesner projections.

Findings

Error bounds between nominal and regularised Riccati flows

Characterization of perturbation effects on Kalman-Bucy distributions

Introduction of new projection models such as Bose-Mesner projections

Abstract

We analyse various perturbations and projections of Kalman-Bucy semigroups and Riccati equations. For example, covariance inflation-type perturbations and localisation methods (projections) are common in the ensemble Kalman filtering literature. In the limit of these ensemble methods, the regularised sample covariance tends toward a solution of a perturbed/projected Riccati equation. With this motivation, results are given characterising the error between the nominal and regularised Riccati flows and Kalman-Bucy filtering distributions. New projection-type models are also discussed; e.g. Bose-Mesner projections. These regularisation models are also of interest on their own, and in, e.g., differential games, control of stochastic/jump processes, and robust control.

Equations

$$\left\{\begin{array}[]{rcl}dX_{t}&=&A\,X_{t}~{}dt~{}+~{}R^{1/2}\,dW_{t}\\ dY_{t}&=&C\,X_{t}~{}dt~{}+~{}\Sigma^{1/2}\,dV_{t}\end{array}\right.$$

$${\hat{X}}_t:=\mathbb{E}(X_t|{\mathcal{F}}_t)\text{mbox}\mathrm{and}{P}_t:=\mathbb{E}\left(\left(X_t-\mathbb{E}(X_t|{\mathcal{F}}_t)\right){\left(X_t-\mathbb{E}(X_t|{\mathcal{F}}_t)\right)}^{\prime }\right)$$

$$d{\hat{X}}_t=A{\hat{X}}_{t}dt+P_t{C}^{\prime }{\Sigma }^{-1}\left(d{Y}_t-C{\hat{X}}_{t}dt\right)\text{mbox}\mathrm{with}{\partial }_t{P}_t=\text{mbox}\mathrm{Ricc}(P_t).$$

$$\text{mbox}\mathrm{Ricc}(Q)=AQ+Q{A}^{\prime }-QSQ+R\text{mbox}\mathrm{with}S:=C^{\prime }{\Sigma }^{-1}C.$$

$$d{\overline{X}}_t=A{\overline{X}}_{t}dt+R^{1/2}d{\overline{W}}_t+{\mathcal{P}}_{{\eta }_t}{C}^{\prime }{\Sigma }^{-1}\left[d{Y}_t-\left(C{\overline{X}}_{t}dt+{\Sigma }^{1/2}d{\overline{V}}_t\right)\right]$$

$${\mathcal{P}}_{{\eta }_t}={\eta }_t\left[(e-{\eta }_t(e))(e-{\eta }_t(e){)}^{\prime }\right]\text{mbox}\mathrm{with}{\eta }_t:=\text{mbox}\mathrm{Law}({\overline{X}}_t|{\mathcal{F}}_t)\text{mbox}\mathrm{and}e(x):=x.$$

$$\left\{\begin{array}[]{rcl}d\xi^{i}_{t}&=&A~{}\xi^{i}_{t}dt+R^{1/2}d\overline{W}_{t}^{i}+p_{t}C^{\prime}\Sigma^{-1}\left[dY_{t}-\left(C\xi^{i}_{t}dt+\Sigma^{1/2}~{}d\overline{V}^{i}_{t}\right)\right]\\ i&=&1,\ldots,N\end{array}\right.$$

$$d{\overline{X}}_t^{\pi }=A{\overline{X}}_t^{\pi }dt+R^{1/2}d{\overline{W}}_t+\pi ({\mathcal{P}}_{{\eta }_t^{\pi }})C^{\prime }{\Sigma }^{-1}\left[d{Y}_t-\left(C{\overline{X}}_t^{\pi }dt+{\Sigma }^{1/2}d{\overline{V}}_t\right)\right]$$

$${\partial }_t{P}_t^{\pi }$$

$$1)\pi =id+\Delta \text{mbox}\mathrm{with}\Delta \approx 0\text{mbox}\mathrm{or}2)\pi \circ \pi =\pi$$

$${\partial }_t{P}_t^{\pi }$$

$$\mathbb{E}\left[\Vert \pi (p_0)-P_0{\Vert }_F^2\right]=\Vert \pi (P_0)-P_0{\Vert }_F^2+\mathbb{E}\left[\Vert \pi (p_0)-\pi (P_0){\Vert }_F^2\right]$$

$$\mathbb{E}\left[\Vert L\odot (p_0-P_0){\Vert }_2\right]\le c{\log }^3(2r)\left[\frac{l}{N}+\sqrt{\frac{l}{N}}\right]\Vert P_0{\Vert }_2$$

$$\begin{array}[]{rcl}\mu(A)&:=&\inf{\{\alpha:\forall x,~{}\langle x,Ax\rangle\leq\alpha\left\|x\right\|_{2}^{2}\}}\\ &=&\lambda_{max}\left(A_{sym}\right)\\ &=&\inf{\{\alpha:\forall t\geq 0,~{}\|\exp{(At)}\|_{2}\leq\exp{(\alpha t)}\}}.\end{array}$$

$$\mu (A)\ge \varsigma (A):=\max \left\{\text{mbox}\mathrm{Re}(\lambda ):\lambda \in \text{mbox}\mathrm{Spec}(A)\right\}$$

$$\left\|A\right\|_{F}^{2}=\mbox{\rm tr}(A^{\prime}A)\qquad\mbox{\rm with the trace operator $\mbox{\rm tr}(\mbox{\LARGE.})$.}$$

$$\Vert A{\Vert }_2^2={\lambda }_{max}(A^{\prime }A)\le \text{mbox}\mathrm{tr}(A^{\prime }A)=\Vert A{\Vert }_F^2\le r\Vert A{\Vert }_2^2.$$

$${\lambda }_{min}(A{A}^{\prime }{)}^{1/2}\Vert B{\Vert }_F\le \Vert AB{\Vert }_F\le {\lambda }_{max}(A{A}^{\prime }{)}^{1/2}\Vert B{\Vert }_F.$$

$$\Vert Q_1^{1/2}-Q_2^{1/2}\Vert \le {\left[{\lambda }_{min}^{1/2}(Q_1)+{\lambda }_{min}^{1/2}(Q_2)\right]}^{-1}\Vert Q_1-Q_2\Vert$$

$$P\ge Q\ge 0\text{mbox}\mathrm{and}A\ge B\ge 0⟹P\odot A\ge Q\odot B.$$

$$\eta (f)=\mathbb{E}(f(Z))=\int f(x)\eta (dx)$$

$${\mathbb{W}}_n({\nu }_1,{\nu }_2)=\inf \left\{\mathbb{E}{\left(\Vert Z_1-Z_2{\Vert }_2^n\right)}^{\frac{1}{n}}\right\}.$$

$$\mbox{\rm Ent}\left(\nu_{1}~{}|~{}\nu_{2}\right):=\int~{}\log\left(\frac{d\nu_{1}}{d\nu_{2}}\right)~{}d\nu_{1}\quad\mbox{if $\nu_{1}\ll\nu_{2}$, and $+\infty$ otherwise.}$$

$${\varphi }_t^{\pi }(Q)\ge {\varphi }_t(Q).$$

$${\partial }_t{P}_t^{\pi }$$

$${\Gamma }_{\pi }(Q)=B_0+B_{1}Q+Q{B}_1^{\prime }+Q{B}_{2}Q+\mathcal{R}(Q)$$

$$\underset{t\ge 0}{\sup }\Vert {\varphi }_t^{\pi }(Q)-{\varphi }_t(Q){\Vert }_2\le c(\delta )ϵ\text{mbox}and\underset{t\ge 0}{\sup }\mathbb{E}{\left[\Vert {\psi }_{0,t}^{\pi }(x,Q)-{\psi }_{0,t}(x,Q){\Vert }_2^{2n}\right]}^{\frac{1}{2n}}\le c(\delta )\sqrt{n}ϵ$$

$${\eta }_{s,t}(x,Q):=\text{mbox}\mathrm{Law}\left({\overline{\psi }}_{s,t}(x,Q)|{\mathcal{F}}_{s,t}\right)\text{mbox}\mathrm{and}{\eta }_{s,t}^{\pi }(x,Q):=\text{mbox}\mathrm{Law}\left({\overline{\psi }}_{s,t}^{\pi }(x,Q)|{\mathcal{F}}_{s,t}\right)$$

$$\text{mbox}\mathrm{Ent}\left({\eta }_{s,t}^{\pi }(x,Q)|{\eta }_{s,t}(x,Q)\right)$$

$${\mathbb{W}}_{2n}\left[{\eta }_{s,t}^{\pi }(x,Q),{\eta }_{s,t}(x,Q)\right]$$

$$\pi(Q)=\operatorname*{\arg\!\min}_{B\in{\cal B}}{\pi\left[(Q-B)(Q-B)^{\prime}\right]}\quad\mbox{\rm for some matrix ring ${\cal B}\subset\mathbb{M}_{r}$}.$$

Full text

1 Introduction

The purpose of this work is to analyse a number of perturbations and projections of Kalman-Bucy [49, 16] semigroups and of the associated (matrix differential) Riccati flow.

The results in this work are of interest in their own mathematical right. However, a prime motivating application for this work is the ensemble Kalman filter (EnKF) [34] and the various “regularisation” methods used to ensure well-posedness of the sample covariance (e.g. sufficient rank) and to “move” the sample covariance closer (in some sense) to the Riccati flow of the true Kalman filter [49, 16]. For example, two common forms of regularisation are covariance inflation-type methods (perturbations) and so-called covariance localisation methods (projections). Covariance inflation is a simple idea that involves adding some positive definite matrix to the sample covariance in order to increase its rank [7]; i.e. more specifically to account for an under-representation of the true variance due to a potentially inferior sample size. Separately, the idea of covariance localization involves multiplying (element-wise) the EnKF sample covariance matrix via Schur (or Hadamard) products with certain sparse “masking” matrices with the intent of reducing spurious long-range correlations and increasing the sample covariance rank [46, 65]. See [42] for an empirical examination of both types of regularisation. In these two cases, choosing the right inflation or localization is non-trivial and numerous ideas exist; e.g. [36, 37, 4, 59, 5, 73, 6]. Other related, and/or more subtle, regularisation methods exist and we will cover more general models in more detail in later sections; see also [43, 70, 3, 78, 74, 47, 75, 45] for related EnKF methodology.

Note that the total literature on EnKF methodology is too broad to cover adequately here. Results on EnKF convergence are recent (relative to this work) and concern, e.g., weak convergence with sample size [57, 63, 55], and stability [60, 79, 80, 31, 29, 17], etc. The articles [80, 62] concern stability and robustness of the EnKF in the presence of specific inflation and localisation methods. The articles [19, 17] study the behaviour of matrix-valued Riccati diffusions that capture the flow of the sample covariance various EnKF implementations; i.e. its moment behaviour (non-asymptotic bias and variance), convergence, central-limit-type behaviour, stability etc.

From a purely mathematical vantage, regularisation amounts to studying various projections and perturbations of the “standard” Riccati flow (viz [49, 16]). The analytical behaviour of general projections and perturbations are a major focus of this study. We consider a broad class of perturbation model. We consider a particular projection model, and a certain class of localizable/diagonalizable systems adapted to these projections; the details are specified later. New ideas concerning projections relevant to the EnKF are also introduced within this class. Given this analysis, we then study the (nonlinear) Kalman-Bucy diffusion [16] and provide a number of contraction-type convergence results between the corresponding perturbed/projected diffusion and the optimal Kalman-Bucy diffusion. We study convergence in the mean-square sense and also in terms of the law of the diffusion.

While methods in data assimilation and ensemble Kalman filtering are the main drivers of this work, the types of perturbations considered herein are more widely relevant: For example, our analysis captures well those perturbations of the “standard” Riccati flow that arise in, e.g., linear quadratic differential games [13, 64, 30], in the control of linear stochastic jump systems [28, 2], in certain robust and H∞ control settings [35, 12], etc; see also the early work of Wonham [83] in linear-quadratic stochastic control. We also highlight the text [1, e.g. Chap. 6] and the references therein. Separately, a specific projected Riccati flow is studied in [23]. Other relevant and related literature considering similar-type projections in estimation theory is given in [71, 61]. Going forward, we primarily rely on EnKF motivators, but we emphasise here that the mathematical development is more broadly applicable.

Further introduction, discussion, and background is given in later subsections with a more technical focus. The organisation of this article is as follows:

Contents

1. 1 Introduction

2. 1.1 Kalman-Bucy diffusions

3. 1.2 Perturbations and projections

4. 1.3 Some background notation

5. 1.4 Statement of the main results

6. 1.5 Some background results

7. 2 Riccati semigroups

8. 2.1 Variational and backward semigroups

9. 2.2 Perturbation-type models

10. 2.3 Projection-type models

11. 3 Kalman-Bucy stochastic flows

12. 3.1 Perturbation-type models

13. 3.2 Projection-type models

14. 4 Some applications

15. 4.1 Variance inflation models

16. 4.2 Block-diagonal localization

17. 4.3 Bose-Mesner projections

18. 4.4 Stein-Shrinkage models

19. 4.5 Mean repulsion models

1.1 Kalman-Bucy diffusions

The notation used throughout this article is introduced later in Section 1.3. However, the set-up in this section is relatively standard. Consider a time homogeneous linear-Gaussian filtering model of the following form

[TABLE]

where $(W_{t},V_{t})$ is an $(r+r^{\prime})$-dimensional standard Brownian motion, $X_{0}$ is a $r$-valued Gaussian random vector (independent of $(W_{t},V_{t})$) with mean $\mathbb{E}(X_{0})$ and covariance matrix $P_{0}$, the symmetric positive definite matrices $R^{1/2}$ and $\Sigma^{1/2}$ are invertible, $A$ is an arbitrary square $(r\times r)$-matrix, $C$ is an arbitrary $(r^{\prime}\times r)$-matrix, and $Y_{0}=0$. Let ${\cal F}_{t}=\sigma\left(Y_{s},~{}s\leq t\right)$ be the $\sigma$-algebra filtration generated by the observations.

It is well-known [16] that the conditional distribution $\eta_{t}$ of the signal state $X_{t}$ given ${\cal F}_{t}$ is a $r$-dimensional Gaussian distribution with a mean and covariance matrix given by

[TABLE]

given by the Kalman-Bucy and the Riccati equations

[TABLE]

In the above display, Ricc stands for the Riccati drift function from $\SS^{+}_{r}$ into $\SS_{r}$ defined for any $Q\in\SS^{+}_{r}$ by

[TABLE]

We now consider the conditional nonlinear McKean-Vlasov type diffusion process

[TABLE]

where $(\overline{W}_{t},\overline{V}_{t},\overline{X}_{0})$ are independent copies of $(W_{t},V_{t},X_{0})$ (thus independent of the signal and the observation path). The notation ${\cal P}_{\eta_{t}}$ stands for the covariance matrix

[TABLE]

We shall call this probabilistic model (1.4) the Kalman-Bucy (nonlinear) diffusion process.

The ensemble Kalman-Bucy filter (EnKF) coincides with the mean-field particle approximation of the nonlinear diffusion process (1.4). To be more precise we let $(\overline{W}^{i}_{t},\overline{V}^{i}_{t},\xi^{i}_{0})_{1\leq i\leq N}$ be $N$ independent copies of $(\overline{W}_{t},\overline{V}_{t},\overline{X}_{0})$. In this notation, a naive EnKF is given by the Mckean-Vlasov type interacting diffusion process

[TABLE]

with the rescaled particle covariance $p_{t}:=\left(1-N^{-1}\right)^{-1}{\cal P}_{\eta^{N}_{t}}$ and where the covariance matrix ${\cal P}_{\eta^{N}_{t}}$ is defined similarly to (1.5) but in terms of the empirical measures $\eta^{N}_{t}:=N^{-1}\sum_{1\leq i\leq N}\delta_{\xi^{i}_{t}}$.

We define the following semigroup notation.

Definition 1.1**.**

We let $\theta_{s,t}(x)$ be the stochastic flow associated with the underlying signal process (1.1). We let $\phi_{s,t}(Q)$ be the semigroup associated with the matrix Riccati equation in (1.2) with (1.3). And we let $\psi_{s,t}(x,Q)$ and $\overline{\psi}_{s,t}(x,Q)$ be the vector stochastic flows associated with the Kalman-Bucy filter and the nonlinear diffusion defined in (1.2) and (1.4), with $s\leq t$ and $(x,Q)\in\mathbb{R}^{r}\times\SS_{r}^{+}$.

We also make the following standing assumption: Throughout this work we take the standard controllability and observability conditions as holding; see Section 1.5.1 for a statement of these conditions, and [49, 8, 16] for a broader discussion and details on controllability and observability in control and filtering theory.

A key feature of any EnKF method, is the sample-based estimation of the solution to the Riccati equation using a collection of interacting Kalman-Bucy filters. Contrary to conventional covariance estimates based on independent random samples, the EnKF is based on interacting samples. These samples are sequentially updated by a noisy observation process through a gain matrix that itself depends on the sample covariance. The corresponding process is highly nonlinear (even when the true signal and observation model is linear). In high dimensions, the interacting particle estimation of the Riccati solution experiences the same difficulties as any conventional sample covariance estimator. For example:

• •

The sample covariance $p_{t}$ is the sample mean of $N-1$ independent unit-rank matrices and has null eigenvalues when $N-1<r$. Thus, in some principal directions, the EnKF is driven solely by the signal diffusion. With unstable signals, the EnKF will exhibit divergence as it is not corrected by the innovation process. In this setting, one cannot design a stable particle sampler of the nonlinear diffusion (1.4) without some kind of regularization.

• •

The estimation of sparse high-dimensional covariance matrices using a small number of independent samples cannot readily be achieved without incorporating some information on the sparsity structure of the desired limit. Several regularization techniques have been developed in the statistics literature; see e.g. [41, 56, 27, 25, 14, 33, 53, 15, 58, 50, 26, 9]. One key common feature is to eliminate (typically long-range) noisy-type empirical correlations when its known that the limiting correlation is null or very small.

1.2 Perturbations and projections

From a pure mathematical position, our model of perturbation or projection is motivated by methodology that replaces the sample covariance $p_{t}$ in (1.6) by some matrix $\pi(p_{t})$, where $\pi:\SS^{+}_{r}\mapsto\SS^{+}_{r}$ is some judiciously chosen mapping. These methods coincide with the mean field particle approximation of the nonlinear diffusion $\overline{X}_{t}^{\pi}$ defined by (1.4) with ${\cal P}_{\eta_{t}}$ replaced by $\pi({\cal P}_{\eta^{\pi}_{t}})$, i.e.,

[TABLE]

where $\eta^{\pi}_{t}=\mbox{\rm Law}(\overline{X}_{t}^{\pi}~{}|~{}{\cal F}_{t})$. The initial state $\overline{X}_{0}^{\pi}$ is a Gaussian random variable with some covariance matrix ${\cal P}_{\eta^{\pi}_{0}}$. We expect the empirical average of the EnKF system associated with (1.7) to converge to the Kalman-Bucy filter defined by (1.2) except with $P_{t}$ replaced by the matrix $\pi(P_{t})$. From the statistical viewpoint, the Kalman-Bucy filter $\widehat{X}^{\pi}:=\mathbb{E}(\overline{X}_{t}^{\pi}\,|\,{\cal F}_{t})$ defined by (1.7) captures the limiting bias of the EnKF empirical mean, introduced by some perturbation and/or projection operator $\pi$. The nonlinear diffusion (1.7) is well posed, and the flow of covariance matrices $P^{\pi}_{t}={\cal P}_{\eta^{\pi}_{t}}$ satisfies

[TABLE]

when $\pi$ is chosen so that (1.8) has a unique positive definite solution; a proof of this assertion is provided in the Appendix. This equation captures the covariance flow of the limiting perturbed/projected Kalman-Bucy filter $\widehat{X}^{\pi}:=\mathbb{E}\left(\overline{X}_{t}^{\pi}\,|\,{\cal F}_{t}\right)$ associated with (1.7). Consequently, (1.8) captures the bias in the limiting EnKF sample covariance as $N\rightarrow\infty$. This perturbed or projected Riccati equation (1.8) is the main object of study in this work.

Note that we focus on the limiting object (1.8). Our analysis holds if one replaces (1.6), or the regularised limiting object (1.7), with regularised versions of the (unregularised) “deterministic” EnKF in [74] (or those in [72, 76]). The deterministic EnKF in [74] swaps $d\overline{V}^{i}_{t}$ in (1.6) with a deterministic adjustment factor. Regularised versions of other EnKF variants may also be considered, if they lead to the same regularised limiting object of interest, i.e. (1.8); e.g. any EnKF “flavour” leaving $\eta^{\pi=id}_{t}$ unchanged is covered. Of course, when studying the perturbed Riccati equation (1.8) alone, our analysis is not even limited to EnKF-type motivation, as noted in the introduction (and again later).

We define the following semigroup notation.

Definition 1.2**.**

Given some mapping $\pi$ from $\SS_{r}^{+}$ into itself, we let $\phi_{s,t}^{\pi}(Q)$, resp. $\psi_{s,t}^{\pi}(x,Q)$ and $\overline{\psi}^{\pi}_{s,t}(x,Q)$ be the semigroup, respectively the stochastic flows associated with the Riccati equation (1.8), respectively the Kalman-Bucy filter and the Kalman-Bucy diffusion associated with the nonlinear model (1.7), with $s\leq t$ and $(x,Q)\in\mathbb{R}^{r}\times\SS_{r}^{+}$.

In the further development we shall distinguish and analyze the two different cases:

[TABLE]

where $id$ stands for the identity mapping.

The first class of model can be thought of as a local perturbation mapping. These mappings are associated to some parameter that describe the level of perturbation. This model includes the variance inflation techniques discussed in Section 4.1 and Stein-Shrinkage models presented in Section 4.4, among others.

The second class of model corresponds to projection-type mappings such as masked projections (or localization methods) discussed in Section 4.2 and projection mappings on Bose-Mesner algebras discussed in Section 4.3.

Later in Section 4.5 we consider mean-repulsion type perturbations, and we highlight how the main results presented in this work can be applied more broadly than implied by (1.9) alone.

We also show later that the first class of model can actually capture most projections considered herein, or more general classes of test-type driving estimators; see the discussion in Section 4.

1.2.1 Discussion: Perturbation-type regularization

Consider the first class of perturbation model in (1.9). Under this model, several variance inflation methods have been proposed in the data assimilation literature as a simple means to address some of these numerical issues [7, 42, 4, 59, 5]. By far the simplest technique is to add an artificial diagonal (positive definite) matrix to the sample covariance matrix $p_{t}$ in (1.6). Another strategy is to consider a general class of Stein-Shrinkage-type perturbations models. These two strategies are discussed in Section 4.1 and Section 4.4.

As an example, in view of (1.7), (1.8), a simple variance inflation method $\pi(Q):=Q+\Delta(Q)$, yields the following Riccati evolution

[TABLE]

Obviously, such artificial inflations introduce an extra bias in the particle estimates delivered by the EnKF (beyond the bias caused by a finite sample size and (nonlinear) interacting particles). In this example, a non-vanishing inflation term would generally be the sole cause of bias in the limiting EnKF empirical mean and covariance as $N\rightarrow\infty$.

Later, we consider more general perturbation mappings that may arise in scenarios outside (ensemble) Kalman filtering such as in differential games, or in the control of linear stochastic jump systems, etc. These applications were briefly referenced in the introduction. These models will capture the preceding perturbation map (1.10) as a special case.

Analysis of any bias-variance relationship trade-off requires one to quantify somewhat these two terms. This work focuses on the bias, in particular as it follows from the mapping $\pi$. For example, with the EnKF, the $\mathbb{L}_{2}$-error estimate at the origin with respect to the Frobenius norm is

[TABLE]

whenever $\mathbb{E}(p_{0})=P_{0}$ and $\mathbb{E}[\pi(p_{0})]=\pi(P_{0})$. Unfortunately, this unbiasedness property is not preserved in time $t>0$, due to the mean-field interactions; i.e. the EnKF estimate $p_{t}$ of $P_{t}$ is biased in any case (e.g. even with $\pi=id$) due to the particle approximation/interaction. We don’t study the bias arising from the mean field approximation here, and our analysis is mostly deterministic and focused on the relevant regularisation mappings. See [19, 17] for a detailed study of the bias (and variance, etc) of a stochastic matrix Riccati diffusion that captures the flow of the (finite $N$) sample covariance in a naive (non-regularised $\pi=id$) EnKF implementation.

The general class of all perturbation-type mappings considered in this work is discussed in Section 2.2 and Section 3.1 (see also Sections 4.1 and 4.4).

1.2.2 Discussion: Projection-type regularization

Consider now the second class of projection models in (1.9). Under the EnKF framework, these projections are often defined in terms of the Hadamard product (a.k.a. Schur product) of the sample covariance matrix with some mask [36, 46]. Here we may approximate such masks with a matrix $L$ of $\{0,1\}$-valued entries. The null entries capture the desired sparsity of the estimate. In the signal processing and data assimilation literature, these projections are often referred to as localization techniques. The study of $\{0,1\}$-valued mask matrices $L$ allows us to make rigorous convergence statements, and these results may act as a proxy for qualitatively understanding the behaviour in more general cases such as those considered in [36, 46]. In the statistics literature, a random matrix given by the Hadamard product $L\odot p_{0}$ associated with some sample covariance $p_{0}$ is called a masked (or banded) sample covariance estimator of some limiting matrix $P_{0}$, see [14, 33, 58, 26].

These projection techniques require the solution of the true unperturbed Riccati equation (the desired limit of the sample covariance) to lie within some class of (at least “approximately”) “band-able” covariance matrices. To avoid the introduction of a huge bias [65], some prior knowledge of the sparsity/correlation structure of the solution to the Riccati equation is typically needed. However, the sparsity structure of a prescribed filtering problem is generally difficult to extract from the signal and sensor models etc. In some cases, the sparsity structure of the matrices $P_{t}$ can be estimated online from the particle model; e.g. see the Isomap algorithm described in [77, 82].

As with the first class of perturbation models, the choice of mapping $\pi$ under the second class of projection model introduces a deterministic bias. For example, in the filtering problem discussed in Section 4.2, $P_{0}$ is a block-diagonal covariance matrix associated with $n$-independent filtering problems. In this case, we have $\pi(P_{0})=L\odot P_{0}=P_{0}$ for some judicious block-diagonal matrix $L$ with $\{0,1\}$-valued entries. With this choice, it also follows that $L\odot P_{t}=P_{t}$. However, as noted before, the EnKF derived (finite) sample covariance matrices are always (randomly) biased due to the (random) particle approximations/interactions, so that $L\odot p_{t}\not=p_{t}$ for any $t>0$. Hence the effect of this projection in practice is to “enforce” some structure on the sample covariance at each time. In the limit $N\rightarrow\infty$ one hopes to recover the property $L\odot p_{t}\rightarrow L\odot P_{t}=P_{t}$.

In the general case, the fluctuations of $L\odot p_{0}$ around its limiting average value $L\odot P_{0}$ depend only on the non-zero entries. More precisely, for any symmetric mask-matrix $L$ with $\{0,1\}$-entries and at most $l$-zeros in each row we have the Levina-Vershynin’s inequality,

[TABLE]

for some finite universal constant $c<\infty$; see [58, 26]. Of course, as before, this relationship is not so nicely preserved in time $t>0$ when comparing $p_{t}$ and $P_{t}$, due to the random particle approximation/interaction which introduces its own bias and fluctuations. Again we point to [19, 17] for a discussion on these random (particle) induced fluctuations.

Another example class of projections discussed in detail in Section 4.3 are orthogonal projections on Bose-Mesner-type cellular algebras w.r.t. the Frobenius norm [20]. These more sophisticated projections are more interesting than those examples in Section 4.2 and can be used to project sample covariance matrices based on the topological/graph structure of the matrices $(A,R,S)$.

The general class of all projection-type mappings considered in this work is discussed in Section 2.3 and Section 3.2; see also Section 4.2 and 4.3 for those examples discussed above.

1.3 Some background notation

This section details some basic notation and terms used throughout the article.

Let $\left\|\mbox{\LARGE.}\right\|_{2}$ be the Euclidean norm on $\mathbb{R}^{r}$, $r\geq 1$. We denote by $\mathbb{M}_{r}$ the set of $(r\times r)$-square matrices with real entries, $\SS_{r}\subset\mathbb{M}_{r}$ the set of $(r\times r)$ real symmetric matrices, and by $\SS_{r}^{+}\subset\SS_{r}$ the subset of symmetric positive (semi)-definite matrices. With a slight abuse of notation, we denote by $Id$ the $(r\times r)$ standard identity matrix (with the size obvious from the context). Given some subsets ${\cal I},{\cal J}\subset\{1,\ldots,r\}$ we set $A_{{\cal I},{\cal J}}=\left(A_{i,j}\right)_{(i,j)\in({\cal I}\times{\cal J})}$ and $A_{{\cal I}}=A_{{\cal I},{\cal I}}$.

Denote by $\lambda_{i}(A)$, with $1\leq i\leq r$, the non-increasing sequence of eigenvalues of a $(r\times r)$-matrix $A$ and let $\mbox{\rm Spec}(A)$ be the set of all eigenvalues. We often denote by $\lambda_{min}(A)=\lambda_{r}(A)$ and $\lambda_{max}(A)=\lambda_{1}(A)$ the minimal and the maximal eigenvalue. We set $A_{sym}:=(A+A^{\prime})/2$ for any $(r\times r)$-square matrix $A$. We define the logarithmic norm $\mu(A)$ of an $(r_{1}\times r_{1})$-square matrix $A$ by

[TABLE]

The above equivalent formulations show that

[TABLE]

where $\mbox{\rm Re}(\lambda)$ stands for the real part of the eigenvalues $\lambda$. The parameter $\varsigma(A)$ is often called the spectral abscissa of $A$. Also note that $A_{sym}$ is negative definite as soon as $\mu(A)<0$. The Frobenius matrix norm of a $(r_{1}\times r_{2})$ matrix $A$ is defined by

[TABLE]

If $A$ is a matrix $(r\times r)$, we have $\left\|A\right\|_{F}^{2}=\sum_{1\leq i,j\leq r}A(i,j)^{2}$. For any $(r\times r)$-matrix $A$, we recall norm equivalence formulae

[TABLE]

For any matrices $A$ and $B$ we also have the estimate

[TABLE]

We also quote a Lipschitz property of the square root function on (symmetric) positive definite matrices. For any $Q_{1},Q_{2}\in\SS_{r}^{+}$

[TABLE]

for any unitary invariant matrix norm (such as the $\mathbb{L}_{2}$-norm or the Frobenius norm). See for instance Theorem 6.2 on page 135 in [44], as well as Proposition 3.2 on page 591 in [81].

The Hadamard-Schur product of two $(r\times r^{\prime})$-matrices $A$ and $B$ of the same size is defined by the matrix $A\odot B$ with entries $(A\odot B)_{i_{1},i_{2}}=A_{i_{1},i_{2}}B_{i_{1},i_{2}}$ for any $1\leq i_{1}\leq r$ and $1\leq i_{2}\leq r^{\prime}$. With a slight abuse of notation, we denote by $J$ the $(r\times r^{\prime})$ Hadamard-Schur identity matrix with all unit entries. By Theorem 17 in [48], we recall that for any symmetric positive semi-definite matrices $(A,B,P,Q)$ we have

[TABLE]

Now, given some random variable $Z$ with some probability measure or distribution $\eta$ and some measurable function $f$ on some product space $\mathbb{R}^{r}$, we let

[TABLE]

be the integral of $f$ w.r.t. $\eta$ or the expectation of $f(Z)$. As a rule any multivariate variable, say $Z$, is represented by a column vector and we use the transposition operator $Z^{\prime}$ to denote the row vector (similarly for matrices; already seen above).

We also need to consider the $n$-th Wasserstein distance between two probability measures $\nu_{1}$ and $\nu_{2}$ on $\mathbb{R}^{r}$ defined by

[TABLE]

The infimum in the above formula is taken over all pairs of random variable $(Z_{1},Z_{2})$ such that $\mbox{\rm Law}(Z_{i})=\nu_{i}$, with $i=1,2$. We denote by $\mbox{\rm Ent}\left(\nu_{1}~{}|~{}\nu_{2}\right)$ the Boltzmann-relative entropy

[TABLE]

1.4 Statement of the main results

In Section 2.2 and Section 2.3 (cf. Theorem 2.4 and formula (2.17)) we will check that

[TABLE]

This property shows that any $\pi$-perturbation or $\pi$-projection of the Kalman-Bucy diffusion induces a larger covariance matrix w.r.t. the Loewner order. This property is one key driving motivation for regularisation in the EnKF literature.

Our first contribution concerns the continuity properties of the first class of perturbation models presented in (1.9) and introduced more formally in Section 2.2. We consider a compact subset $\Pi$ of continuous mappings $\pi~{}:~{}\SS_{r}^{+}\mapsto~{}\SS_{r}^{+}$ equipped with the uniform norm induced by the $\mathbb{L}_{2}$-norm on $\SS_{r}^{+}$. We let $B(\delta)$ be a $\delta$-ball around the identity mapping. For example, consider (1.7), (1.8) and suppose further that

[TABLE]

with the quadratic positive mapping $\Gamma_{\pi}$ defined by

[TABLE]

for some matrices $(B_{0},B_{1},B_{2})\in\SS_{r}^{3}$ with $B_{2}\leq S$ and $\varpi:=\sup_{Q\in\SS^{+}_{r}}\|{\cal R}(Q)\|_{2}<\infty~{}\Rightarrow~{}{\cal R}(Q)\leq\varpi~{}Id.$ This model captures, e.g., simple inflation models like (1.10), and Stein-Shrinkage methods like those discussed in Section 4.4. This model also captures those perturbations relevant in, e.g., linear-quadratic differential games, control of stochastic jump processes, robust control theory, etc.

This mapping $\Gamma_{\pi}$ already hints that the analysis of the semigroups $\phi^{\pi}_{t}$ is a delicate mathematical problem, since it cannot be deduced directly from that of the Riccati flow $\phi_{t}$. By the Cauchy-Lipschitz theorem, the existence and the uniqueness of the flow of matrices $\phi^{\pi}_{t}(Q)$ for any starting covariance matrix $Q$ is ensured by the local Lipschitz property of the drift function $\mbox{\rm Ricc}^{\pi}$, on some open interval that may depend on $Q$. The existence of global solutions on the real line is not ensured as the quadratic term may induce a blow up on some finite time horizon.

In this setting, our first main result concerns the first class of perturbation models presented in (1.9), and takes the following mildly informal form.

Theorem 1**.**

Assume that the filtering problem is observable and controllable. In this situation, under some regularity conditions, there exists some $\delta>0$ such that for any $\epsilon<\delta$, any $\pi\in B(\epsilon)$, and any $n\geq 1$ we have the uniform estimates

[TABLE]

for some finite constant $c(\delta)$ whose values only depend on the parameter $\delta$.

A precise statement of this result is given in Theorem 2.6 and Theorem 3.2; e.g. with clarification of the required regularity conditions. The proof of the Riccati estimates in the l.h.s. of (1.15) is provided in Section 2.2.2, dedicated to the boundedness and the robustness properties of Riccati semigroups (cf. Theorem 2.6; see also [18] for further discussion on these robustness and related results). The proof of the r.h.s. estimates in (1.15) is provided in Section 3.1 dedicated to the continuity properties of Kalman-Bucy stochastic flows (cf. Theorem 3.2).

The preceding theorem concerns time-uniform bounds on the mean and the covariance of the Kalman-Bucy flows. Our second objective, given the first class of perturbations, is to quantify the difference between the conditional distributions of the nonlinear Kalman-Bucy diffusion,

[TABLE]

where ${\cal F}_{s,t}=\sigma(Y_{u},s\leq u\leq t)$ denotes the $\sigma$-field generated by the observations from time $s$ to the time horizon $t$. By construction $\overline{\psi}_{s,t}$ and $\overline{\psi}_{s,t}^{\,\pi}$ are time-varying Ornstein-Uhlenbeck-type diffusions (linear stochastic differential equations) [16] and consequently $\eta_{s,t}(x,Q)$ and $\eta_{s,t}^{\pi}(x,Q)$ are both Gaussian distributions. Our next main result informally takes the following form.

Theorem 2**.**

Under the assumptions of Theorem 1, for any $n\geq 1$, we have the almost sure relative entropy and Wasserstein distance estimates

[TABLE]

for some constant $c<\infty$ that depends on the system and observation matrices.

The proof of these estimates, with a more precise description of the constant $c$, is provided in Section 3.1; e.g. see the precise statement of these results in Theorem 3.3 and Theorem 3.5.

The impact of these two theorems is illustrated in Section 4.1 and Section 4.4 in terms of the variance inflation and the Stein-Shrinkage methods common in the data assimilation literature.

Our second contribution concerns the continuity properties of the second class of projection mappings presented in (1.9) and discussed further in Section 2.3. We assume that $\pi$ is some positive map from $\mathbb{M}_{r}$ into itself, of the form

[TABLE]

From the geometrical viewpoint, these orthogonal projections map the set $\SS_{r}^{+}$ into the set of matrices with the same sparsity structure as the matrices in the ring ${\cal B}$. These projection techniques are unbiased when the covariance graph of the filtering model (reflecting the sparsity structure of the matrices $P_{t}$) is defined in terms of the same association scheme. Thus, the optimal use of these projections requires some prior knowledge on the sparsity structure of the solution to the Riccati equation. This is a special class of projection model differing somewhat from the typical localization used in the EnKF literature; e.g. see [36, 46]. However, under the particular chosen class of projection, explicit and rigorous convergence results are possible when the correlation structure is well-enough adapted to the projection. Heuristically, these results may act as a proxy to gain intuitive, or qualitative, insight into the behaviour of more practical localization implementations [46]; e.g. and can be taken in combination with the first class of perturbation model for this purpose.

A prototype model satisfying these conditions are orthogonal projections onto the set of block-diagonal matrices ${\cal B}={\cal M}_{r[1]}\oplus\ldots\oplus{\cal M}_{r[n]}\subset\mathbb{M}_{r}$, with $r=\sum_{1\leq q\leq n}r[q]$. Another important class of models satisfying the above conditions are orthogonal projections on Bose-Mesner-type cellular algebras w.r.t. the Frobenius norm [20]. These more sophisticated projections are interesting and can be used to project sample covariance matrices based on the topological/graph structure of the matrices $(A,R,S)$.

See Section 4.2 for applications to block-diagonal masking matrices and Section 4.3 for further discussion on Bose-Mesner projections; e.g. Section 4.3.4 provides an explicit solution of the Riccati equation as soon as the matrices $(A,R,S)$ and the initial condition belong to some Bose-Mesner algebra.

In this context, our third main result takes the following mildly informal form.

Theorem 3**.**

Assume that the filtering problem is observable and controllable and assume that $(A,A^{\prime},S,R)\in{\cal B}$. In this situation we have

[TABLE]

for any $(x,Q)\in(\mathbb{R}^{r}\times\SS_{r}^{+})$ and $t\geq 0$. In addition, there exists some $\rho>0$ such that for any $Q\in\SS_{r}^{+}$ and any time horizon $t\geq 0$ we have the local exponential-Lipschitz inequality

[TABLE]

for some finite constant $c_{Q}$ whose values only depend on $\|Q\|_{2}$.

The result in (1.16) is stated precisely as Theorem 2.12 and is covered also in Section 3.2. The estimate (1.17) is stated precisely in Theorem 2.13; see also the corollaries in Section 2.3.1.

The relationship (1.16) shows that the set ${\cal B}$ is stable w.r.t. the $\pi$-projected Riccati flow. The exponential estimate (1.17) shows that, for any initial condition, the Kalman-Bucy stochastic flow as well as the $\pi$-projected Riccati flow converges to the set ${\cal B}$ as the time horizon $t$ tends to $\infty$.

Last, but not least, Theorem 3 allows one to transfer, without further work, all the exponential contraction inequalities developed in [16], dedicated to the stability properties of Kalman-Bucy diffusions.

1.5 Some background results

1.5.1 Observability, controllability and the steady-state Riccati equation

We assume that $(A,R^{1/2})$ is a controllable pair and $(A,C)$ is observable in the sense that

[TABLE]

have rank $r$. With $R$ positive definite as assumed here (and common in filtering problems), the controllability condition follows immediately. We consider the observability and controllability Gramians $({\cal O}_{t},{\cal C}_{t}({\cal O}))$ and $({\cal C}_{t},{\cal O}_{t}({\cal C}))$ associated with the triplet $(A,R,S)$ and defined by

[TABLE]

Given the rank assumptions on (1.18), there exists some parameters $\upsilon,\varpi^{o,c}_{\pm},\varpi^{c}_{\pm}({\cal O}),\varpi^{o}_{\pm}({\cal C})>0$ such that

[TABLE]

as well as

[TABLE]

The parameter $\upsilon$ is often called the interval of observability-controllability. By Theorem 4.4 in [16], for any $t\geq\upsilon$ and any $Q\in\mathbb{S}_{r}^{+}$ we have the uniform estimates

[TABLE]

When (1.19) is satisfied, we say that a triplet $(A,R,S)$ satisfy the Gramian condition for some parameters $\upsilon,\varpi^{o,c}_{\pm}>0$. These conditions ensure the existence and the uniqueness of a positive definite fixed-point matrix $P$ solving the so-called algebraic Riccati equation

[TABLE]

Importantly, in this case, the matrix difference $A-PS$ is asymptotically stable (Hurwitz stable) even when the signal matrix $A$ is unstable [54, Theorems 9.12, 9.15]. More relaxed conditions (i.e. detectability and stabilisability) for a stabilising solution (perhaps only positive semi-definite) to exist are discussed widely in the literature; see [51, 66, 54] and the convergence results in [52, 22].

1.5.2 Exponential and Kalman-Bucy semigroup estimates

The transition matrix associated with a smooth flow of $(r\times r)$-matrices $A:u\mapsto A_{u}$ is denoted by

[TABLE]

for any $s\leq t$, with ${\cal E}_{s,s}=Id$, the identity matrix. Equivalently in terms of the fundamental solution matrices ${\cal E}_{t}(A):={\cal E}_{0,t}(A)$ we have ${\cal E}_{s,t}(A)={\cal E}_{t}(A){\cal E}_{s}(A)^{-1}$.

The following technical lemma provides a pair of semigroup estimates of the state transition matrices associated with a sum of drift-type matrices.

Lemma 1.3** ([16]).**

Let $A~{}:~{}u\mapsto A_{u}$ and $B~{}:~{}u\mapsto B_{u}$ be some smooth flows of $(r\times r)$-matrices. For any $s\leq t$ and any matrix norm $\|\cdot\|$ we have

[TABLE]

for some positive constant $\alpha_{A}$ and some parameter $\omega_{A}$.

For any $s\leq t$ and $Q\in\SS_{r}^{+}$ we set

[TABLE]

When $s=0$ sometimes we write $E_{t}(Q)$ instead of $E_{0,t}(Q)$. In this notation we have

[TABLE]

For any $s\leq u\leq t$ and $Q\in\SS_{r}^{+}$ we set

[TABLE]

Also observe that

[TABLE]

For any $s\leq u\leq t$ and any $Q\in\SS_{r}^{+}$ we have

[TABLE]

Observe that the Riccati equation is time-homogeneous so that

[TABLE]

By Proposition 4.3 in [16] we have

[TABLE]

for some constant $\kappa$ whose values doesn’t depend on the time parameter nor on $Q$. We also have the following contraction result.

Theorem 1.4** ([16]).**

For any $Q_{1},Q_{2}\in\SS^{+}_{r}$ and for any $t\geq 0$ we have the local contraction inequality

[TABLE]

for some rate $\nu>0$, and some finite non-decreasing functions $\kappa_{E}(q_{1}),\kappa_{E}(q_{1},q_{2})$, $\kappa_{\phi}(q_{1},q_{2})<\infty$.

2 Riccati semigroups

2.1 Variational and backward semigroups

We let ${\cal L}(\SS_{r},\SS_{r})$ be the set of bounded linear functional from $\SS_{r}$ into itself, and equipped with the Frobenius norm. A mapping $\phi:\SS^{+}_{r}\mapsto\SS^{+}_{r}$ is Fréchet differentiable at some $Q_{1}\in\SS_{r}^{+}$ if there exists a continuous linear functional $\partial\phi(Q_{1})\in{\cal L}(\SS_{r},\SS_{r})$ such that

[TABLE]

For instance the first-order Frechet-derivative of the Riccati quadratic drift function

[TABLE]

defined in (1.3) is given for any $(Q_{1},Q_{2})\in(\SS_{r}^{+}\times\SS_{r})$ by the formula

[TABLE]

Lemma 2.1**.**

For any $t\geq 0$ the mapping $Q\mapsto\phi_{t}(Q)$ is Fréchet differentiable and for any $(Q_{1},Q_{2})\in(\SS_{r}^{+}\times\SS^{+}_{r})$ we have the formulae

[TABLE]

Proof.

Using the decomposition

[TABLE]

we have

[TABLE]

We end the proof of the first assertion using the Lipschitz property (1.25). The proof of the lemma is completed. ∎

We have the following backward flow and first-order variational result that will be used subsequently, but which is also of interest in its own right.

Proposition 2.2**.**

For any $Q\in\SS_{r}^{+}$ and any $0\leq s\leq t$ we have

[TABLE]

In addition, the first-order variational equation associated with the Riccati equation is given by the composition formula

[TABLE]

Proof.

For any $Q\in\SS_{r}^{+}$ we have

[TABLE]

On the other hand, we have

[TABLE]

for some finite constant $c_{Q}$ whose values only depends on $\|Q\|_{F}$. Using Lemma 2.1 this yields

[TABLE]

This implies that

[TABLE]

from which we conclude that

[TABLE]

Finally, by Lemma 2.1 and (2.1) we have

[TABLE]

This ends the proof of the proposition. ∎

2.2 Perturbation-type models

2.2.1 First and second order perturbations

We consider perturbation-type distortions in (1.8) of the first type in (1.9). Formally, consider (1.8) and (1.14) and the class of perturbation mappings $\Gamma_{\pi}$ in (1.14) with the hypothesis

[TABLE]

for some given matrices $(B_{0},B_{1},B_{2})\in\SS_{r}^{3}$ such that $B_{2}\leq S$, and a uniformly bounded (symmetric) remainder term

[TABLE]

In this situation, the $\pi$-Riccati drift function $\mbox{\rm Ricc}^{\pi}$ in (1.8) takes the form

[TABLE]

with

[TABLE]

and the matrices

[TABLE]

Definition 2.3**.**

We let $\phi_{\pi,t}$, resp. $\phi^{\pi}_{t}$ be the Riccati flows associated with the drift function $\mbox{\rm Ricc}_{\pi}$ and resp. $\mbox{\rm Ricc}^{\pi}$. We consider the observability and the controllability Gramians $({\cal O}_{\pi,t},{\cal C}_{\pi,t}({\cal O}))$ and $({\cal C}_{\pi,t},{\cal O}_{\pi,t}({\cal C}))$ associated with the triplet $(A_{\pi},R_{\pi},S_{\pi})$.

We also let $\Xi_{\pi}$ be the mapping from $\SS_{r}$ into itself defined by

[TABLE]

We also set

[TABLE]

We consider the following condition,

[TABLE]

We recall that this condition ensures the existence and the uniqueness of a positive definite fixed-point matrix $P_{\pi}$ solving the so-called algebraic Riccati equation

[TABLE]

In addition, the matrix difference $A_{\pi}-P_{\pi}S_{\pi}$ is asymptotically stable.

Our first objective is to analyze the existence and the uniqueness of the flow $\phi^{\pi}_{t}$.

Theorem 2.4**.**

Assume $\mbox{\rm(H)}_{0}$ and $\mbox{\rm(H)}_{1}$. For any $t\geq(\upsilon\vee\upsilon_{\pi})$ and $Q\in\SS^{+}_{r}$,

[TABLE]

Proof.

By (1.20) and $\mbox{\rm(H)}_{1}$ we have the uniform estimates

[TABLE]

We let $E_{\pi,t|s}(Q)$ be the transition semigroups defined as $E_{t|s}(\phi^{\pi}_{s}(Q))$ by replacing $(A,\phi_{t})$ by $(A_{\pi},\phi_{\pi,t})$. In this notation, the proof (2.6) is a direct consequence of the backward perturbation formulae

[TABLE]

as well as

[TABLE]

That is, the l.h.s. estimate in (2.6) is a direct consequence of (1.20) and the relationship $\phi_{t}(Q)\leq\phi^{\pi}_{t}(Q)\leq\phi_{\pi,t}(Q)$ following from (2.7) and (2.8). The r.h.s. estimate in (2.6) follows obviously from the above.

To check (2.8) we use the interpolating path

[TABLE]

By Proposition 2.2 we have

[TABLE]

This ends the proof of (2.8). The proof of (2.7) follows the same arguments, thus it is skipped. This ends the proof of the theorem. ∎

The next lemma compares the semigroups $\phi_{\pi,t}(Q)$ and $\phi_{t}(Q)$ when the matrices $(A_{\pi},R_{\pi},S_{\pi})$ are close to $(A,R,S)$.

Lemma 2.5**.**

Assume $\mbox{\rm(H)}_{0}$ and $\mbox{\rm(H)}_{1}$. For any $t\geq 0$ and $Q\in\SS^{+}_{r}$ we have

[TABLE]

as well as

[TABLE]

The proof of this lemma follows the same arguments as the proof of Theorem 2.4; thus it is skipped. Observe that

[TABLE]

and

[TABLE]

2.2.2 Robustness theorems

We equip the set ${\cal C}(\SS_{r}^{+},\SS_{r}^{+})$ of continuous mappings $\pi~{}:~{}\SS_{r}^{+}\mapsto\SS_{r}^{+}$ with the uniform norm

[TABLE]

Let $\Pi\subset{\cal C}(\SS_{r}^{+},\SS_{r}^{+})$ be a compact subset, and let $t>0$ be some fixed time horizon. For any $\delta>0$, we let $B(\delta)$ be the $\delta$-ball around the identity mapping; that is

[TABLE]

We consider the following continuity condition

[TABLE]

Assume that $\mbox{\rm(H)}_{0}$ and $\mbox{\rm(H)}_{1}$ are met. Importantly, in this situation we have

[TABLE]

We check this claim using the fact that

[TABLE]

Of course, it is important to note that we can take $\mbox{\rm(H)}_{2}$ without $\mbox{\rm(H)}_{0}$, and consider directly just the flow $\phi_{\pi,t}$. The main objective of this section is to prove the following robustness theorem.

Theorem 2.6**.**

Let $(A_{\pi},R_{\pi},S_{\pi})$ be a collection of matrices satisfying condition $\mbox{\rm(H)}_{2}$. In this case, there exists some $\delta>0$ such that for any $\pi\in B(\delta)$, any horizon $t\geq 0$ and any $Q\in\SS_{r}^{+}$ we have

[TABLE]

for some finite constants $\chi_{1}(\delta)$ and $\chi_{2}(\delta,\|Q\|_{2})$, whose values only depend on the parameters $\delta$, and $(\delta,\|Q\|_{2})$ respectively. In particular we have

[TABLE]

Note that whenever we take $\mbox{\rm(H)}_{0}$ as holding, then we take $\mbox{\rm(H)}_{2}$ as being compatible in the definition of $(A_{\pi},R_{\pi},S_{\pi})$ as given in (2.4). This allows us to compare $\phi^{\pi}_{t}$ and $\phi_{\pi,t}$. In this case, (2.11) means this theorem guarantees the boundedness of those perturbation models satisfying (1.8), (1.14) and $\mbox{\rm(H)}_{0}$. See also [18] for a refined/corrected discussion on this and related results.

Note also however, that $\mbox{\rm(H)}_{2}$, and Theorem 2.6, capture a broader class of perturbation model than (1.8), (1.14) and $\mbox{\rm(H)}_{0}$ alone. In particular, $\mbox{\rm(H)}_{2}$ is simply concerned with direct perturbations of the original $(A,R,S)$ system matrices in the Riccati operator (1.3).

The proof of the preceding theorem relies on the following proposition.

Proposition 2.7**.**

When $\mbox{\rm(H)}_{2}$ is met, for any $\alpha\in]0,1]$ there exists some $\delta>0$ such that the matrices $(A_{\pi},R_{\pi},S_{\pi})$ indexed by mappings $\pi$ in the $\delta$-ball $B(\delta)$ satisfy the Gramian condition with a common interval of observability-controllability $\upsilon_{\pi}=\upsilon$ and some parameters

[TABLE]

and well as

[TABLE]

We already quote a direct consequence of Theorem 2.4 and Proposition 2.7.

Corollary 2.8**.**

Assume $\mbox{\rm(H)}_{2}$. In this situation, for any $\alpha\in]0,1]$ there exist some $\delta>0$ such that for any $\pi\in B(\delta)$, any $Q\in\SS_{r}^{+}$ and any $t\geq 0$ we have the common uniform estimates

[TABLE]

If $\mbox{\rm(H)}_{0}$ also holds, then we know additionally that $\phi_{t}(Q)\leq\phi^{\pi}_{t}(Q)\leq\phi_{\pi,t}(Q)$.

We have already studied the ordering between $\phi_{t}$, $\phi^{\pi}_{t}$ and $\phi_{\pi,t}$ in Theorem 2.4 when $\mbox{\rm(H)}_{0}$ holds. But if we take $\mbox{\rm(H)}_{2}$ as holding alone (i.e. forget $\mbox{\rm(H)}_{0}$ and $\phi^{\pi}_{t}$), then we cannot compare $\phi_{\pi,t}(Q)$ with $\phi_{t}(Q)$ in the same way. To be more specific, $\phi_{\pi,t+\upsilon}(Q)\geq\phi_{t+\upsilon}(Q)$ holds when $\Xi_{\pi}(Q)\geq 0$, and $\phi_{\pi,t+\upsilon}(Q)\leq\phi_{t+\upsilon}(Q)$ holds when $\Xi_{\pi}(Q)\leq 0$. To check this simply see (2.9).

The proof of the above Proposition 2.7 relies on the next couple of comparison lemmas of interest on their own.

Lemma 2.9**.**

Let $V_{1},V_{2}\in\SS_{r}^{+}$ be a couple of definite positive matrices s.t. $V_{1}\geq V_{2}$. We set

[TABLE]

for some $(U_{1},U_{2})\in\mathbb{M}_{r}^{2}$. Assume that $Q_{2}\geq q_{2}~{}Id$, for some $q_{2}>0$. With $U_{2}$ invertible we have

[TABLE]

with

[TABLE]

Proof.

We set

[TABLE]

Observe that

[TABLE]

In this situation, we have

[TABLE]

Observe that

[TABLE]

On the other hand, we have

[TABLE]

This implies that

[TABLE]

from which we conclude that

[TABLE]

This yields the estimate

[TABLE]

with

[TABLE]

This ends the proof of the lemma.∎

Lemma 2.10**.**

Let ${\cal U},{\cal V}$ be a pair of bounded functions from $[0,t]\times\Pi$ into $\SS_{r}^{+}$. We consider the integral mappings

[TABLE]

Let $\pi_{1},\pi_{2}\in\Pi$ be such that

[TABLE]

Also assume that the flow of matrices ${\cal U}_{s}(\pi_{2})$ are invertible for any $s\in[0,t]$ and they satisfy the following Lipschitz inequality

[TABLE]

for some finite constant $\mbox{\rm lip}_{t}({\cal U})$. In this situation, for any $\epsilon\in]0,1]$ there exists some parameter $\delta=\delta(t,\epsilon,\pi_{2})>0$ such that

[TABLE]

The proof of this lemma follows the same arguments as those in the proof of Lemma 2.9. For the convenience of the reader a detailed proof of the lemma is given in the Appendix.

Now we come to the proof of Proposition 2.7.

Proof of Proposition 2.7:

We assume that for any $\epsilon\in]0,1]$ there exists some $\delta>0$ such that

[TABLE]

We apply Lemma 2.10 to the functions

[TABLE]

with $(\pi_{1},\pi_{2})=(\pi,id)$ and the time horizon $t=\upsilon$. For any $\epsilon_{1}\in]0,1]$ there exists some parameter $\delta_{1}=\delta(\epsilon_{1},\upsilon)$ such that

[TABLE]

We assume that for any $\epsilon\in]0,1]$ there exists some $\delta>0$ such that

[TABLE]

We apply Lemma 2.10 to the functions

[TABLE]

with $(\pi_{1},\pi_{2})=(id,\pi)$ and the time horizon $t=\upsilon$. From previous estimates we have

[TABLE]

By Lemma 2.10 for any $\epsilon_{2}\in]0,1]$ we can choose $\delta=\delta(\epsilon_{1},\epsilon_{2},\upsilon)$ such that

[TABLE]

This shows that

[TABLE]

In the same vein we prove the estimates of the Gramians ${\cal O}_{\pi,\upsilon}({\cal C})$ and ${\cal C}_{\pi,\upsilon}({\cal O})$. This ends the proof of the proposition.∎

We are now in position to prove Theorem 2.6

Proof of Theorem 2.6:

By Corollary 2.8, there exist some $\delta>0$ s.t. for any $\pi\in B(\delta)$ we have the uniform estimate

[TABLE]

for some finite constant $\chi_{1}(\delta)$ whose values only depend on $\delta$. We have

[TABLE]

for some constant $\chi_{2}(\delta)$ whose values only depend on $\delta$. Combining (1.24) with (2.9) we have

[TABLE]

We collect constants in the first term with $\chi_{2}(\delta,\|Q\|)$ and in the second term (via some notation abuse) with $\chi_{1}(\delta)$. This ends the proof of the first assertion.

To check the last assertion we simply let $t\uparrow\infty$. More precisely, observe that

[TABLE]

This implies, using (1.25) and the first assertion of this theorem, that

[TABLE]

Letting $t\rightarrow\infty$ we end the proof of the desired estimate. This ends the proof of the theorem.∎

See also [18] for further discussion on these robustness and related results.

2.3 Projection-type models

We consider projection-type mappings in (1.8) of the second type in (1.9). Let $\pi$ be some positive map from $\mathbb{M}_{r}$ into itself; that is $\pi(\SS_{r}^{+})\subseteq\SS^{+}_{r}$. We first assume the matrices $(A,R,S)$ satisfy

[TABLE]

and we let ${\cal B}\subset\mathbb{M}_{r}$ be a given matrix ring. Also assume that the pair $(\pi,{\cal B})$ satisfies the following orthogonality property:

[TABLE]

In this situation, we have

[TABLE]

This shows that $\pi$ can be interpreted as a $\pi$-orthogonal projection

[TABLE]

In addition, we have the Cauchy-Schwartz inequality

[TABLE]

Whenever ${\cal B}$ is closed by transposition we have

[TABLE]

The identity mapping $\pi=id$ and the set ${\cal B}=\mathbb{M}_{r}$ clearly satisfies the above properties.

The prototype of a non-trivial pair $({\cal B},\pi)$ satisfying $\mbox{\rm(H)}_{3}$ are orthogonal projections $\pi=\mbox{\rm proj}_{{\cal B}}$ (w.r.t. the Frobenius norm) onto cellular (a.k.a. coherent) algebras; that is a sub-algebra of matrices which are additionally closed under the Hadamard-Schur product and contains the identity elements $Id$ and $J$, where $J$ stands for the matrix with all ones entries. Up to a unitary change of basis, these projections can be reformulated in terms of block-diagonal matrices [11]. By [21, page 57], a sub-algebra of $\SS_{r}$ is a Bose-Mesner algebra [20] of some association scheme if and only if it contains $I$ and $J$, and it is closed under the Hadamard-Schur product. This shows that cellular sub-algebras of $\SS_{r}$ coincide with the Bose-Mesner algebras (of some association scheme). We refer to Section 4.3 for a detailed discussion on these models.

The set ${\cal B}={\cal M}_{r[1]}\oplus\ldots\oplus{\cal M}_{r[n]}\subset\mathbb{M}_{r}$ (with $r=\sum_{1\leq q\leq n}r[q]$) of block-diagonal matrices with null entries outside the blocks is also a matrix ring which is closed under the Hadamard-Schur product w.r.t. any matrix in $\mathbb{M}_{r}$; but ${\cal B}$ is not a cellular algebra since $J\not\in{\cal B}$. The orthogonal projection from $\mathbb{M}_{r}$ onto this ${\cal B}$ is given by

[TABLE]

In the above display, $J_{i}$ stands for the $i$-th block unit matrix w.r.t. the Hadamard-Schur product; that is the $(r[i]\times r[i])$-square matrix with all unit entries. It is readily checked that $({\cal B},\pi)$ satisfies condition $\mbox{\rm(H)}_{3}$. We refer to Section 4.2 for a discussion on these models.

We let $\phi^{\pi}_{t}$ be the $\pi$-Riccati semigroup defined in Section 1.4. By (2.8) we have the domination property

[TABLE]

In contrast with the second order approximation models discussed in Section 2.2.1 these projection techniques don’t depend on some perturbation index that quantifies the distance between $\pi$ and the identity mapping.

When $(\pi(A),\pi(S),\pi(R))\not=(A,R,S)$ we can replace $(A,R,S)$ by their projections $(A_{\pi},R_{\pi},S_{\pi})$. In this case, $\phi_{\pi,t}$ is the Riccati semigroup associated with the drift function $\mbox{\rm Ricc}_{\pi}(Q)$ defined simply by $\mbox{\rm Ricc}(Q)$ with $(A,R,S)$ replaced by $(A_{\pi},R_{\pi},S_{\pi})$. The difference between $\phi_{\pi,t}$ and $\phi_{t}$ can be analyzed as in Theorem 2.6. It is not possible to ensure that $\phi_{\pi,t}$ is arbitrarily close to $\phi_{t}$ without some continuity conditions.

Section 4.4 discusses a way to combine these projection-type models with the perturbation-type models discussed in Section 2.2.1

In the latter development of Section 2.3.1 we will provide exponential concentration inequalities that ensure the $\pi$-projected Riccati flows converge exponentially fast to the solution of the (nominal, Kalman-Bucy) Riccati equation, viz [49, 16], as the time horizon tends to $\infty$, and as soon as condition $\mbox{\rm(H)}_{3}$ is met. Speaking somehow loosely we shall show that

[TABLE]

Definition 2.11**.**

We let $\varphi^{\pi}_{t}(Q)$ be the flow of the projected Riccati equation

[TABLE]

The next theorem shows that the flows $\varphi^{\pi}_{t}$ and $\phi^{\pi}_{t}$ coincide with the $\pi$-projection of the Riccati flow $\pi\circ\phi_{t}$ as soon as we start from an initial state $Q$ that satisfies $\pi(Q)=Q$ and $(A,R,S)\in{\cal B}^{3}$. It also provides an explicit description of the flow $\phi^{\pi}_{t}$ in terms of $\phi_{t}$ and $\pi$ when $\mbox{\rm(H)}_{3}$ is satisfied.

Theorem 2.12**.**

Assume $\mbox{\rm(H)}_{3}$ and recall $\mbox{\rm Ricc}^{\pi}$ defined in (1.8). For any time horizon $t\geq 0$ we have the formula

[TABLE]

as well as the semigroup commutation properties

[TABLE]

In addition, we have the formula

[TABLE]

Proof.

Recall that $(A,A^{\prime},R,S)\in{\cal B}$. Since ${\cal B}$ is a matrix ring we have

[TABLE]

or equivalently

[TABLE]

Also observe that

[TABLE]

and thus

[TABLE]

Now, we also have

[TABLE]

This implies that

[TABLE]

This yields

[TABLE]

and by the uniqueness of the solution of the Riccati equation we conclude that

[TABLE]

We also have

[TABLE]

which implies that

[TABLE]

This completes the proof of (2.18).

Now, we have

[TABLE]

This implies that

[TABLE]

The l.h.s. estimate in (2.19) is a consequence of the domination property (2.17). This ends the proof of the theorem. ∎

2.3.1 Exponential contraction inequalities

We continue with the projection-type models and $\mbox{\rm(H)}_{3}$ holding.

Theorem 2.13**.**

For any $Q_{1},Q_{2}\in\SS^{+}_{r}$ and for any $t\geq 0$ we have

[TABLE]

some finite constant $\kappa_{\phi^{\pi}}(Q_{1},Q_{2})<\infty$ whose values only depend on $\left(\|Q_{1}\|_{2},\|Q_{2}\|_{2}\right)$. This implies the existence of an unique fixed point $P^{\pi}=\phi^{\pi}_{t}(P^{\pi})$ with $\pi(P^{\pi})=P$. In addition, for any $Q\in\SS^{+}_{r}$ and for any $t\geq 0$ we have

[TABLE]

Proof.

We have

[TABLE]

This implies that

[TABLE]

Using (1.24) we find that

[TABLE]

To estimate the second term, we use the decomposition

[TABLE]

Combining (1.24) with (1.26) we find that

[TABLE]

To prove (2.21) we recall from Theorem 2.12 that $\pi\circ\phi^{\pi}_{t}=\phi^{\pi}_{t}\circ\pi$. This implies that

[TABLE]

If we set $Q_{1}=Q_{2}$ we obtain (2.21). This ends the proof of the theorem. ∎

Combining (2.20) with the fact that $\phi^{\pi}_{t}\circ\pi=\phi_{t}\circ\pi$ and $\pi(P^{\pi})=P$ we readily prove the following estimate.

Corollary 2.14**.**

For any $Q\in\SS^{+}_{r}$ and for any $t\geq 0$ we have

[TABLE]

In addition, we have

[TABLE]

The estimate (2.23) is a direct consequence of (1.25) and (2.22). Replacing $Q$ by $P=P^{\pi}$ in (2.22) we obtain the following exponential decays to equilibrium.

Corollary 2.15**.**

For any $Q\in\SS^{+}_{r}$ and for any $t\geq 0$ we have the local Lipschitz estimate

[TABLE]

This yields the uniform estimate

[TABLE]

Combining (1.26) with Corollary 2.15 we prove the following local Lipschitz contraction.

Corollary 2.16**.**

For any $0\leq s\leq t$ and any $Q\in\SS_{r}^{+}$

[TABLE]

some finite constant $\kappa_{E,\pi}(Q,P)<\infty$ whose values only depend respectively on $(\|P\|_{2},\|Q\|_{2})$.

3 Kalman-Bucy stochastic flows

3.1 Perturbation-type models

We consider the perturbation models discussed in Section 2.2. We set

[TABLE]

with

[TABLE]

where $\chi_{2}(\cdot,\cdot)<\infty$ is introduced in Theorem 2.6 and $\nu>0$ and $\kappa_{E}(\cdot)<\infty$ are defined in (1.24).

Recall the semigroup and stochastic flow notation defined in Section 1.4. The first lemma in this section concerns the convergence of the perturbed Kalman-Bucy filter to the true underlying signal process, both in a mean-square sense and in terms of actual sample paths.

Lemma 3.1**.**

Assume $\mbox{\rm(H)}_{0}$ and $\mbox{\rm(H)}_{2}$ are satisfied. For any $\epsilon>0$ there exists some parameter $0<\delta<\epsilon$ such that for any $\pi\in B(\delta)$, $0\leq s\leq t$, $Q\in\SS_{r}^{+}$ and any $n\geq 1$ we have

[TABLE]

In addition, the conditional probability of the following event

[TABLE]

given the state variable $X_{s}$ is greater than $1-e^{-z}$.

Proof.

We have

[TABLE]

In the above display, $t\in[s,\infty[\mapsto M_{s,t}^{\pi}$ stands for the $r$-dimensional martingale

[TABLE]

with angle bracket

[TABLE]

This yields the formula

[TABLE]

with the exponential semigroup $E^{\pi}_{u,t|s}(Q)$ defined for any $s\leq u\leq t$ by

[TABLE]

We have the decomposition

[TABLE]

By (1.24) we have

[TABLE]

By Theorem 2.6 there exists some $\delta>0$ such that for any $\pi\in B(\delta)$

[TABLE]

In addition, we have

[TABLE]

Applying Lemma 1.3 we find that

[TABLE]

For any $\epsilon>0$ we choose $\epsilon>\delta>\delta^{\prime}>0$ and so that for any $\pi\in B(\delta^{\prime})$

[TABLE]

Following the proof of Lemma 5.3 in [16], for any $n\geq 1$ we have

[TABLE]

The end of the proof of the first assertion is now easily completed. The proof of the exponential concentration inequality follows the same line of argument as the proof of Theorem 5.2 in [16]. This ends the proof of the lemma. ∎

The next three theorems concern convergence of the $\pi$-perturbed Kalman-Bucy filter/diffusion to the true, optimal, Kalman-Bucy filter [16]. The first concerns the stochastic flow of the two filters themselves, while the second two theorems concern the associated (conditional) distributions.

Theorem 3.2**.**

Assume $\mbox{\rm(H)}_{0}$ and $\mbox{\rm(H)}_{2}$ are satisfied. In this situation, there exists some parameter $\delta>0$ such that for any $0<\epsilon<\delta$, $\pi\in B(\epsilon)$, $0\leq s\leq t$, $Q\in\SS_{r}^{+}$, and any $n\geq 1$ we have

[TABLE]

for some finite constants ${\chi}(\delta,Q)$ whose value only depends on the parameters $(\delta,\|Q\|_{2})$.

Proof.

We have

[TABLE]

This implies that

[TABLE]

with the $r$-dimensional martingale $t\in[s,\infty[\mapsto M^{\pi}_{s,t}$ defined by

[TABLE]

This implies that

[TABLE]

Arguing as in the proof of Lemma 3.1, there exists some $0<\epsilon<\delta$ such that for any $\pi\in B(\epsilon)$

[TABLE]

and

[TABLE]

By the generalized Minkoswki iequality, we have

[TABLE]

This implies that

[TABLE]

By Lemma 3.1 we have

[TABLE]

This yields the estimate

[TABLE]

with

[TABLE]

and

[TABLE]

Following the proof of Lemma 5.3 in [16], for any $n\geq 1$ we have

[TABLE]

with

[TABLE]

This yields

[TABLE]

This ends the proof of the theorem.∎

Theorem 3.3**.**

Assume $\mbox{\rm(H)}_{0}$ and $\mbox{\rm(H)}_{2}$ are satisfied. In this situation, for any $s+\upsilon\leq t$, and any $Q\in\SS_{r}^{+}$ we have

[TABLE]

Proof.

The Boltzmann relative entropy of $\eta_{s,t}^{\pi}(x,Q)$ w.r.t. $\eta_{s,t}(x,Q)$ is given by the formula

[TABLE]

By Corollary 2.8, for any $t\geq s+\upsilon$ we have

[TABLE]

In addition, there exists some $\delta>0$ s.t. for any $\pi\in B(\delta)$

[TABLE]

This implies that

[TABLE]

The last assertion is a consequence of the following lemma applied to $A=I-\phi_{s,t}(Q)^{-1}\phi^{\pi}_{s,t}(Q)$.

Lemma 3.4**.**

For any $(r\times r)$-matrix $A$ we have

[TABLE]

Proof.

For any $n\geq 1$ we have

[TABLE]

Using the well-known trace formulae

[TABLE]

we conclude that

[TABLE]

The last assertion comes from the inequality

[TABLE]

which is valid for any $u\in[0,1/2[$. This ends the proof of the lemma.∎

To take the final step in the proof of the theorem we note that $\phi^{\pi}_{s,t}(Q)\geq\phi_{s,t}(Q)$ implies

[TABLE]

This ends the proof of the theorem.∎

Theorem 3.5**.**

Assume $\mbox{\rm(H)}_{0}$ and $\mbox{\rm(H)}_{2}$ are satisfied. For any $Q\in\SS_{r}^{+}$, and for $t\geq s+\upsilon$ we have the almost sure Wasserstein estimate

[TABLE]

In addition, for any $n\geq 1$ and any $t\geq s+\upsilon$ we have

[TABLE]

Proof.

The $\mathbb{L}_{2}$-Wasserstein distance between the Gaussian distributions $\eta_{s,t}^{\pi}(x,Q)$, and $\eta_{s,t}(x,Q)$ is given by

[TABLE]

A proof of this formula can be found in [39, 69]. We assume that

[TABLE]

is the principal square root of the positive definite matrix $\phi_{s,t}(Q)^{1/2}\phi_{s,t}^{\pi}(Q)\phi_{s,t}(Q)^{1/2}\geq 0$. Also observe that

[TABLE]

as soon as $t\geq s+\upsilon$. The last estimate is a consequence of Theorem 2.4.

Observe that

[TABLE]

This implies that

[TABLE]

Using (1.12) we have

[TABLE]

By Corollary 2.8 we conclude that

[TABLE]

This ends the proof of the first assertion.

Observe that

[TABLE]

and

[TABLE]

where $Z$ stands for an $r$-dimensional Gaussian random variable with unit covariance matrix, and ${\phi}_{t-s}(x,Q)^{1/2}$ stands for the principal square root of ${\phi}_{t-s}(x,Q)$. Combining (1.12) with Theorem 2.4 for any $n\geq 1$ and any $t\geq s+\upsilon$ we have

[TABLE]

To check the last assertion, we use Stirling approximation to prove that

[TABLE]

This ends the proof of the theorem. ∎

3.2 Projection-type models

We consider the projection models discussed in Section 2.3. The semigroup commutation properties (2.18) already imply that

[TABLE]

Since $\pi(P)=P=P_{\pi}=\pi(P_{\pi})$ we the steady state Kalman-Bucy diffusions coincide; that is we have that

[TABLE]

By Theorem 2.12 we have

[TABLE]

This implies that

[TABLE]

Thus, we have the decompositions

[TABLE]

and

[TABLE]

These formulae show that the convergence analysis of both $\psi_{s,t}^{\pi}(x,Q)-\psi_{s,t}(x,Q)$ and $\overline{\psi}_{s,t}^{\pi}(x,Q)-\overline{\psi}_{s,t}(x,Q)$ to [math], as the time horizon $(t-s)\uparrow\infty$, reduces exactly to the stability properties of the Kalman-Bucy diffusion discussed in the article [16]. We point to this detailed study [16] for the exact Kalman-Bucy convergence results.

4 Some applications

4.1 Variance inflation models

We let $\Pi:=\{\pi_{\epsilon}\,:\,\epsilon\in[0,1]\}$ be the set of mappings

[TABLE]

indexed by $\epsilon\in[0,1]$ and a given reference matrix $T\geq 0$. In this situation, the $\delta$-balls around the identity mapping are given for any $\delta\leq 1$ by the compact sets

[TABLE]

Conditions $\mbox{\rm(H)}_{0}$ and $\mbox{\rm(H)}_{1}$ are clearly met with

[TABLE]

To check $\mbox{\rm(H)}_{2}$ we observe that

[TABLE]

In this situation Theorem 2.6 yields the following corollary.

Corollary 4.1**.**

There exists some $\delta\in[0,1]$ such that for any $\epsilon\in[0,\delta]$ and for any time horizon $t\geq 0$ and any $Q\in\SS_{r}^{+}$ we have

[TABLE]

for some finite constant $\chi_{1}(\delta)$, resp. $\chi_{2}(\delta,\|Q\|)$, whose values only depend on the parameter $\delta$, resp. on $(\delta,\|Q\|)$. In addition, for any $\epsilon\in[0,\delta]$ we have

[TABLE]

4.2 Block-diagonal localization

Assume the covariance matrices associated with the Kalman filter in (1.2) satisfy the property,

[TABLE]

In words, the coordinates of the signal have been arranged so that the $\iota$-long (or longer) range interactions between the state coordinates are null. The above condition is met if and only if the matrices $P_{t}$ are block-diagonal. Since the state variables are Gaussian, this property is equivalent to the fact that the state block components are block-two-by-two marginally independent. In this case, the signal-observation process $(X_{t},Y_{t})=(X_{t}[k],Y_{t}[k])_{1\leq k\leq n}$ defined in (1.1) can be decomposed into $n$-independent $(r[k]\times r^{\prime}[k])$-dimensional filtering problems $(X_{t}[k],Y_{t}[k])$ of the form

[TABLE]

with $r=\sum_{1\leq i\leq n}r[i]$. In this elementary case, the resulting Kalman-Bucy filter and the associated Riccati equation collapse to $n$ independent evolution equations. In this case, the drift and the sensor matrices $(A,C)$, as well as the covariance matrices $(R,\Sigma)$ and $P_{t}$ are block-diagonal matrices of appropriate dimensions.

Now observe that the sample covariance matrices $p_{t}(i,j)$ are generally non-null, even if $P_{t}(i,j)=0$. To mask these noisy entries, we use a localization mapping given in (2.16). It is readily checked that the mapping $\pi$ satisfies the orthogonality condition $\mbox{\rm(H)}_{3}$ discussed in (2.15) with the cellular algebra ${\cal B}={\cal M}_{r[1]}\oplus\ldots\oplus{\cal M}_{r[n]}$. With a little extra work, we can also check that

[TABLE]

with $r^{\star}:=\vee_{1\leq k\leq n}{r(k)}$.

The central idea behind these mask-regularisations is to transform a given sample covariance matrix $p$ into some covariance matrix with the same sparsity pattern as the limiting covariance $P$; or in practice, to mask spurious “long-range” correlations that are (almost) null in the true covariance. This idea is relevant in numerous applications of the EnKF in which state-space interaction and signal observations are mostly local, and a kind-of “decay-of-correlation” effect is present; see [46, 42, 73].

One difficulty is ensuring the mask-matrix $L$ is positive definite so that the projection $L\odot p$ is a positive map. In the block-diagonal model discussed above this property is clearly satisfied. In more general situations, several strategies can be underlined. The first one is to design mask-matrices as linear combinations $L=\sum_{i=1}^{n}l_{i}~{}z_{i}~{}z_{i}^{\prime}$ of unit rank vectors $z_{i}$, with $l_{i}\geq 0$.

4.3 Bose-Mesner projections

We introduce the Bose-Mesner algebra and relevant projections and applications here. For a more thorough discussion on Bose-Mesner algebras and their application in statistical and quantum physics, combinatorics, coding, graph theory, and statistical covariance analysis (more particularly in experimental designs) we refer to the seminal article of Bose-Mesner [20], the ones of Nelder [67, 68], the more recent articles [32, 24, 40], as well as the books [21, 10].

4.3.1 Association schemes

We set ${\cal I}=\{1,\ldots,r\}$ the index set of the coordinates of the signal. Let ${\cal P}=\cup_{0\leq q\leq n}{\cal P}_{q}$ be an $n$-partition of the product set ${\cal I}^{2}$ such that

• •

The associated classes ${\cal P}_{q}$ are symmetric for any $0\leq q\leq n$, and ${\cal P}_{0}:=\{(i,i)~{}:~{}i\in{\cal I}\}$.

• •

For any $0\leq q_{1},q_{2}\leq n$, there exists some integer $w^{q}_{q_{1},q_{2}}$ (the parameters of the scheme; a.k.a. parameters of the first kind or the structural constants) such that

[TABLE]

These association schemes can be interpreted as a partition of the edges/arcs of a complete graph (with vertex set ${\cal I}$) into $n$ classes, often thought of as color classes. In this representation, there is a loop at each vertex and all the loops receive the same [math]-th color. The number of triangles with a fixed arc-base with color $q$ and the other two arcs with colors $q_{1}$ and $q_{2}$ is a number $w^{q}_{q_{1},q_{2}}$ that doesn’t depend on the choice of the arc-base. Each vertex $i$ is contained in exactly $v_{q}$ arcs with color $q$. The number $v_{q}$ is called the valency of the relation induced by ${\cal P}_{q}$. The parameters $w^{q}_{q_{1},q_{2}}=w^{q}_{q_{2},q_{1}}$ are called the parameters of the scheme (a.k.a. parameters of the first kind or the structural constants).

For each $1\leq q\leq n$ we let $B_{q}$ be the adjacency matrix; that is

[TABLE]

We also have

[TABLE]

This shows that $B_{q}$ has exactly $v_{q}$ non-zero entries in every row and every column. Since for any $q_{1}\not=q_{2}$ we have

[TABLE]

the set ${\cal B}$ is also closed w.r.t. the Hadamard product and contains $I,J$. Thus, the set

[TABLE]

is an associative commutative algebra called the Bose-Mesner algebra of the association scheme. Notice that ${\cal B}$ is also a matrix $\star$-algebra (i.e. closed by matrix multiplication, the transposition, addition and the scalar multiplication). These special cases of finite dimensional $\mathbb{C}^{\star}$-algebra are unitarily equivalent to block-diagonal matrices. By a theorem of Von Neumann we also mention that the orthogonal projection on any matrix $\star$-algebra is a positive map.

An illustration when $n=2$ and $r=6$ is provided by

[TABLE]

In this case we have $B_{1}^{2}=2Id+B_{1}=2B_{0}+B_{1}$ and $B_{1}B_{2}=0=B_{2}B_{1}$.

4.3.2 Minimal orthogonal projections

The commuting matrices $B_{q}$ are simultaneously diagonalizable, ${\cal B}$ has a basis of minimal orthogonal idempotents $D_{i}$; that is, we have that

[TABLE]

Without any loss of generality we can choose $D_{0}=r^{-1}J$. The matrices $D_{q}$ are called the minimal idempotents of the algebra ${\cal B}$. In addition, the column vectors $D_{i,1}\ldots,D_{i,r}$ of $D_{i}$ are the eigenvectors of any matrix in ${\cal B}$. The eigenvector spaces ${\cal D}_{i}=\mbox{\rm Span}(D_{i,1}\ldots,D_{i,r})$ are mutually orthogonal and every vector $u\in\mathbb{R}^{r}$ can be expressed uniquely as $u=\sum_{0\leq q\leq n}u_{i}$ with $u_{i}\in{\cal D}_{i}$ (notice that ${\cal D}_{0}$ is the $1$-dimensional space of constant vectors). Also notice that the dimension of ${\cal D}_{i}$ equals to the rank of $D_{i}$, which is equal to the trace of $D_{i}$ (since all non-zero eigenvalues of $D_{i}$ are equal to $1$).

In particular, we have

[TABLE]

where $\lambda_{k}(B_{q})$ stands for the $k$-th eigenvalue of $B_{q}$. Further details on these simultaneous diagonalization can be found in [11].

The orthogonal projection of a matrix $Q$ on ${\cal B}$ is given by the formulae

[TABLE]

To check condition $\mbox{\rm(H)}_{3}$ we observe that

[TABLE]

This yields

[TABLE]

For any matrix $M$ we have

[TABLE]

This implies that

[TABLE]

This shows that the orthogonal projection is a positive map from the algebra of square matrices into itself. In addition, it is trace-preserving and unital in the sense that

[TABLE]

Last, but not least, using the decomposition

[TABLE]

as soon as $Q\in\SS_{r}^{+}$. Working a little harder, we check that

[TABLE]

4.3.3 Distance regular graphs

Another prototype of Bose-Mesner algebra are distance regular graphs. Given a connected graph ${\cal G}=({\cal V},{\cal E})$ with vertex set ${\cal V}$ and arc/edges set ${\cal E}$, we let $\rho(i,j)$ be the path-length distance between two vertices $i,j\in{\cal V}$. Let

[TABLE]

be the sphere of radius $q$. The graph ${\cal G}$ is distance regular if and only if we have

[TABLE]

for some parameters $w^{q}_{q_{1},q_{2}}$. In other words, for every two vertices $(i,j)$ at distance $q$ there are precisely $w^{q}_{q_{1},q_{2}}$ vertices in the graph at distance $q_{1}$ from $i$ and $q_{2}$ from $j$.

In these settings, the matrices

[TABLE]

are called the distance matrices ($B_{0}=Id$, $B_{1}$ the adjacency matrix, and so on). In this situation, the association scheme is given by the partition

[TABLE]

In addition we have $w^{q_{1}}_{q_{2},1}=0$ for any $q_{1}\not=0$, and $q_{2}\not=\{q_{1}-1,q_{1},q_{1}+1\}$. If we set

[TABLE]

then we have

[TABLE]

and

[TABLE]

This shows that the adjacency matrix $B_{1}$ generates ${\cal B}$ (i.e. the matrices $B_{q}$ can be written as polynomials of degree $q$ in $B_{1}$), so that the eigenvalues $\left(\lambda_{k}(B_{1})\right)_{1\leq k\leq d}$ of $B_{1}$ are mutually distinct.

4.3.4 Riccati solvers

A given matrix $Q$ belongs to ${\cal B}$ if and only if it is constant within each block. To check this claim, we observe that

[TABLE]

In other words, the matrix is constant within each block. When $r=r^{\prime}$, then $(\pi(A),\pi(S),\pi(R))=(A,R,S)$ is satisfied as soon as $(A,R,C,\Sigma^{-1})\in{\cal B}$.

We further assume that $(A,R,S)\in{\cal B}$ and we set

[TABLE]

Let $P_{0}=\pi(P_{0})=\sum_{0\leq q\leq n}~{}\alpha_{q}(0)~{}D_{q}$ be some covariance matrix in ${\cal B}$. By Theorem 2.12 we have

[TABLE]

In addition, we have

[TABLE]

This implies that

[TABLE]

When $s_{q}\not=0\not=r_{q}$ this collection of Riccati equations take the form

[TABLE]

with the couple of roots

[TABLE]

The solutions of the above equations are given by the formulae:

[TABLE]

4.4 Stein-Shrinkage models

Stein-Shrinkage models are an extension of the variation inflation model to parameters $\epsilon=\epsilon(Q)$ and target-type matrices $T=T(Q)$ that both may depend on the matrix $Q$. These models are defined by the formula

[TABLE]

for some function $Q\mapsto\epsilon(Q)\in[0,1]$ and some mapping $T$ from $\SS^{+}_{r}$ into itself. It is not within scope of this article to review all the relevant covariance matrix estimators encountered in the statistics literature fitting this general model. We will just illustrate this model with three important and currently used approximations:

• •

Mask matrix estimates are associated with mappings $T$ defined by $T(Q):=L\odot Q$ with a matrix $L$ of the form

[TABLE]

• •

Maximum likelihood type estimates are associated with mappings $T$ defined by

[TABLE]

for some $\alpha>0$, some mask matrix $L$ [15, 25, 50, 53], and some matrix norm $\|\mbox{\LARGE.}\|$ on $\SS^{+}_{r}$.

• •

Nyström estimates are associated with mappings $T$ defined by

[TABLE]

where $\{1,\ldots,r\}={\cal P}\cup{\cal P}^{c}$ stands for a partition of the index coordinate set and $L_{{\cal P}^{c}}$ stands for the mask matrix defined by

[TABLE]

At the level of the sample covariance matrices $p_{0}$, the matrix $T(p_{0})$ is obtained by taking the sample covariance matrix associated with projection ${\cal T}_{{\cal V}}(\zeta_{l})$ of the state particle vectors

[TABLE]

onto the vector space ${\cal V}_{{\cal P}}$ of $\mathbb{R}^{N}$ spanned by the random vectors

[TABLE]

More precisely, if we set

[TABLE]

then we have that

[TABLE]

The proof of this bias property and related variance estimates can be found in [9]. For the convenience of the reader a proof of the last assertion is provided in the Appendix.

For mask type mappings of the form (4.2), condition $\mbox{\rm(H)}_{0}$ is satisfied by first letting

[TABLE]

To ensure the uniform estimate $\sup_{Q\in\SS^{+}_{r}}\|{\cal R}(Q)\|_{2}<\infty$ holds we use Gershgorin’s theorem to show that

[TABLE]

This yields

[TABLE]

When $l_{\iota}(Q)$ is too large, the quadratic perturbation may have some destabilizing effects. To avoid these issues we assume that $\epsilon(Q)$ is chosen so that

[TABLE]

for some $\epsilon_{1}\in[0,1]$, and some threshold $\epsilon_{2}>0$. In this case, condition $\mbox{\rm(H)}_{1}$ is also met with

[TABLE]

Arguing as in the end of Section 4.1 we have

[TABLE]

Now we can consider the set

[TABLE]

for some given parameter $\delta$ and the just described mappings $\pi_{\epsilon_{1},\epsilon_{2}}$ given by

[TABLE]

The associated $\delta$-balls around the identity mapping are given in this case by the compact sets

[TABLE]

for any $\delta\leq 1$.

More generally, the Stein-Shrinkage models discussed above can be extended without further work to general mappings of the following form

[TABLE]

where $T$ stands for some mapping from $\SS_{r}^{+}$ into itself such that

[TABLE]

for some mapping $Q\in\SS_{r}^{+}\mapsto l_{T}(Q)\in[0,\infty[$. Further examples of such mappings include the Bose-Mesner projections $T(Q)=\mbox{\rm proj}_{{\cal B}}(Q)$ discussed in Section 4.3 and which can be seen to fit this model via the trace operator in (4.1).

In this general setting, Theorem 2.6 yields the following corollary.

Corollary 4.2**.**

There exists some $\rho\in[0,1]$ such that for any $(\epsilon_{1},\epsilon_{2})\in([0,1]\times[\delta,\delta^{-1}])$ with $\epsilon_{1}\leq\rho~{}\epsilon_{2}$, for any $Q\in\SS_{r}^{+}$ and any time horizon $t\geq 0$ we have

[TABLE]

for some finite constant $\chi_{1}(\rho)$, resp. $\chi_{2}(\rho,\|Q\|)$, whose values only depend on the parameter $\delta$, resp. on $(\rho,\|Q\|)$. In addition, for any $\epsilon_{1}\leq\rho~{}\epsilon_{2}$ we have

[TABLE]

This section illustrates how our first class of perturbation-type model captures most projection-type mappings; and consequently those results relevant to perturbation-type mappings are applicable to projection-type models (but not vice-versa).

4.5 Mean repulsion models

The preceding subsections were concerned with perturbation and projecting mappings $\pi$ that directly fell within the class of models defined by (1.9). We also illustrated how the first class of perturbation-type model captures most projection-type mappings considered in (1.9).

In this subsection we illustrate that our main result (viz. Theorem 2.6 and $\mbox{\rm(H)}_{2}$) on the robustness and boundedness of perturbed Riccati semigroups, captures a larger class of perturbation-type models than those simply defined by the condition $\mbox{\rm(H)}_{0}$ and (1.14). Of course, Theorem 2.6 also applies under the more constrained condition $\mbox{\rm(H)}_{0}$ as a special case, and $\mbox{\rm(H)}_{0}$ is still of specific interest in, e.g., the variance inflation and Stein-Shrinkage-type models discussed in the preceding subsections. However, $\mbox{\rm(H)}_{0}$ is not satisfied by the perturbation scheme considered in this subsection. Nevertheless, $\mbox{\rm(H)}_{2}$ is satisfied, and thus Theorem 2.6 still applies.

As their name indicates, mean repulsion models are defined by adding an extra repulsion term around the sample averages in the nonlinear diffusion (1.4). Consider the nonlinear diffusion

[TABLE]

where $T_{1}\,:\,\SS_{r}^{+}\mapsto\mathbb{M}_{r}$ stands for some mapping and $T_{2}$ some given matrix.

A key feature of this class of mean repulsion models is that their ${\cal F}_{t}$-conditional projections coincide with the Kalman-Bucy filter, only their conditional covariance matrices are altered.

To describe the Riccati equation associated with this class of nonlinear diffusions we observe that

[TABLE]

Thus, the covariance evolution equation is given by the Riccati equation

[TABLE]

For instance, choosing

[TABLE]

for some $(\epsilon_{1},\epsilon_{2})$ such that $(\epsilon_{1}+\epsilon_{2})>-1/2$ we find that

[TABLE]

We let $\phi_{\epsilon,t}$ be the Riccati semigroup associated with the above equation, with $\epsilon=(\epsilon_{1},\epsilon_{2})\in\Pi=[0,1]^{2}$. Theorem 2.6 yields the following corollary.

Corollary 4.3**.**

There exists some $\delta\in[0,1]$ such that for any $\epsilon=(\epsilon_{1},\epsilon_{2})\in[0,\delta]^{2}$ and for any time horizon $t\geq 0$ and any $Q\in\SS_{r}^{+}$ we have

[TABLE]

for some finite constant $\chi_{1}(\delta)$, resp. $\chi_{2}(\delta,\|Q\|)$, whose values only depend on the parameter $\delta$, resp. on $(\delta,\|Q\|)$. In addition, if $P_{\epsilon}=\phi_{\epsilon,t}(P_{\epsilon})$ is the fixed point of $\phi_{\epsilon,t}$, then for any $\epsilon\in[0,\delta]$ we have

[TABLE]

Appendix

Proof of formula (1.8)

Let $\phi^{\pi}_{s,t}$ be the semigroup of equation (1.8). Also let $\overline{X}_{t}^{\pi}$ be the time non-homogeneous diffusion given by the equation

[TABLE]

with the $r$-valued martingale

[TABLE]

with covariation matrix

[TABLE]

We have

[TABLE]

This implies that the conditional expectations $\widehat{X}_{t}^{\pi}=\mathbb{E}(\overline{X}_{t}^{\pi}~{}|~{}{\cal F}_{t})$ are given by the formula

[TABLE]

Equivalently, we have

[TABLE]

from which we prove that

[TABLE]

This implies that the covariation matrices

[TABLE]

don’t depend on the observation process, and they satisfy the equation

[TABLE]

Recalling that $\phi^{\pi}_{0,t}(Q)$ is the Riccati semigroup of the equation (1.8) we have

[TABLE]

We conclude that

[TABLE]

where $\eta^{\pi}_{t}=\mbox{\rm Law}(\overline{X}_{t}^{\pi}~{}|~{}{\cal F}_{t})$. This ends the proof of (1.8). See also [38, page 242] (among numerous other sources) for the related covariance flow of a Kalman filter with an arbitrary gain matrix.

Proof of Lemma 2.10

Condition (2.13) implies that

[TABLE]

from which we conclude that

[TABLE]

We also have

[TABLE]

Observe that

[TABLE]

with the flow of matrices

[TABLE]

We set

[TABLE]

In this notation, using the fact that

[TABLE]

we find that

[TABLE]

from which we conclude that

[TABLE]

The inequality in (4.8) implies that

[TABLE]

On the other hand we have

[TABLE]

This yields the estimate

[TABLE]

Combining (4.7) with (4.9), for any $\epsilon>0$ there exists some $\delta(t,\epsilon,\pi_{2})>0$ such that

[TABLE]

This ends the proof of the lemma. ∎

Proof of the bias estimate (4.6)

Observe that if $Z\sim\mathcal{N}(0,Q)$ is Gaussian, then the conditional distribution of $Z_{{\cal P}^{c}}=(Z_{k})_{k\in{\cal P}^{c}}$ given $Z_{{\cal P}}=(Z_{k})_{k\in{\cal P}}$ is again a centred Gaussian with covariance matrix

[TABLE]

where $Q_{{\cal P}}^{-}$ stands for the Moore-Penrose pseudo-inverse of $Q_{{\cal P}}$. The matrix $T_{{\cal P}}(Q)$ can be seen as the Schur complement of $Q_{{\cal P}}$ in $Q$. This shows that

[TABLE]

In this notation we have

[TABLE]

We let $g$ be the matrix

[TABLE]

Also let $g^{-}=(g^{i,j})_{1\leq i,j\leq s}$ be the pseudo-inverse of $g$. The orthogonal projection of a vector $\zeta_{l}$ with $l\not\in{\cal P}$ is given by

[TABLE]

[TABLE]

Given ${\cal V}$, the $N$ random vectors $\zeta^{i}_{{\cal P}^{c}}=(\zeta_{k}^{i})_{k\not\in{\cal P}}\in\mathbb{R}^{r-s}$, with $1\leq i\leq N$ are independent random vectors in $\mathbb{R}^{N}$ with mean

[TABLE]

and covariance matrix

[TABLE]

This implies that for any $k,l\not\in{\cal P}$ we have

[TABLE]

On the other hand, we have

[TABLE]

Taking the expectation we find that

[TABLE]

This shows that

[TABLE]

This ends the proof of (4.6).∎