Multi-agent estimation and filtering for minimizing team mean-squared error

TL;DR

This paper introduces the concept of team mean-squared error (MTMSE) for multi-agent estimation, deriving closed-form solutions and demonstrating improved performance over traditional methods in vehicle platoon scenarios.

Contribution

The paper develops the MTMSE framework for multi-agent estimation, providing closed-form solutions and recursive filtering algorithms that outperform existing methods.

Findings

MTMSE estimates outperform MMSE and consensus Kalman filtering.

Closed-form expressions for MTMSE are derived for linear systems.

Simulation shows significant improvement in vehicle distance estimation.

Abstract

Motivated by estimation problems arising in autonomous vehicles and decentralized control of unmanned aerial vehicles, we consider multi-agent estimation and filtering problems in which multiple agents generate state estimates based on decentralized information and the objective is to minimize a coupled mean-squared error which we call \emph{team mean-square error}. We call the resulting estimates as minimum team mean-squared error (MTMSE) estimates. We show that MTMSE estimates are different from minimum mean-squared error (MMSE) estimates. We derive closed-form expressions for MTMSE estimates, which are linear function of the observations where the corresponding gain depends on the weight matrix that couples the estimation error. We then consider a filtering problem where a linear stochastic process is monitored by multiple agents which can share their observations (with delay) over a communication graph. We derive expressions to recursively compute the MTMSE estimates. To illustrate the effectiveness of the proposed scheme we consider an example of estimating the distances between vehicles in a platoon and show that MTMSE estimates significantly outperform MMSE estimates and consensus Kalman filtering estimates.

Tables

Table 1. TABLE I : Comparison of the size and performance of the three information structures for the values of parameters of Sec.  IV-B and λ = 100 𝜆 100 \lambda=100 .

Info structure		Dimension of local info		Performance $J_T^{\ast }/\lambda$
		$i\in \{1,4\}$	$i\in \{2,3\}$
IS0 :	${\left\{I_i\left(t\right)\right\}}_{i\in N}$	6	8	180.46
IS1 :	${\left\{I_i^{\left(1\right)}\left(t\right)\right\}}_{i\in N}$	3	4	193.72
IS2 :	${\left\{I_i^{\left(2\right)}\left(t\right)\right\}}_{i\in N}$	3	3	252.09

Equations

$$y_i=x+v_i,v_i\sim \mathcal{N}(0,{\sigma }^2),$$

$$\begin{gathered} \text{mathds}E[(x-{\hat{z}}_1{)}^2+(x-{\hat{z}}_2{)}^2]+\lambda \text{mathds}E\left[{\left(x-\frac{{\hat{z}}_1+{\hat{z}}_2}{2}\right)}^2\right] \\ =\text{mathds}E\left[{\left[\begin{array}{c}x-{\hat{z}}_1\\ x-{\hat{z}}_2\end{array}\right]}^{⊺}\left[\begin{array}{cc}1+\frac{\lambda }{4}& \frac{\lambda }{4}\\ \frac{\lambda }{4}& 1+\frac{\lambda }{4}\end{array}\right]\left[\begin{array}{c}x-{\hat{z}}_1\\ x-{\hat{z}}_2\end{array}\right]\right], \end{gathered}$$

$${\hat{z}}_i=g_i^{\mathrm{mmse}}(y_i):=\text{mathds}E[x\mid y_i]=\frac{1}{1+{\sigma }^2}{y}_i,$$

$$J^{\mathrm{mmse}}=J(g^{\mathrm{mmse}}_{1},g^{\mathrm{mmse}}_{2})=2\Bigl{(}\frac{\sigma^{2}}{1+\sigma^{2}}\Bigr{)}\Bigl{(}1+\frac{\lambda}{4}\cdot\frac{1+2\sigma^{2}}{1+\sigma^{2}}\Bigr{)}.$$

$${\hat{z}}_i=g_i^{\mathrm{lin}}(y_i)=F{y}_i$$

$$J^{\mathrm{lin}}=J(g^{\mathrm{lin}}_{1},g^{\mathrm{lin}}_{2})=(2+\lambda)(1-F)^{2}+2\Bigl{(}1+\frac{\lambda}{4}\Bigr{)}F^{2}\sigma^{2}$$

$$F=\frac{1}{1+\frac{1+\lambda /4}{1+\lambda /2}{\sigma }^2}=\frac{1}{1+\alpha {\sigma }^2},$$

$$J^{\mathrm{lin}}=(2+\lambda )\frac{\alpha {\sigma }^2}{1+\alpha {\sigma }^2}.$$

$$\Delta :=\frac{J^{\mathrm{mmse}}-J^{\mathrm{lin}}}{J^{\mathrm{lin}}}\approx \frac{1}{2}\cdot \frac{{\sigma }^2}{(1+{\sigma }^2{)}^2},$$

$$c(x,{\hat{z}}_1,\dots ,{\hat{z}}_n)=\sum _{i\in N}\sum _{j\in N}(L_{i}x-{\hat{z}}_i{)}^{⊺}{S}_{ij}(L_{j}x-{\hat{z}}_j).$$

$$c(x,\hat{z})=(Lx-\hat{z}{)}^{⊺}S(Lx-\hat{z}),$$

$$S=\left[\begin{array}{ccc}{S}_{11}& \cdots & S_{1n}\\ ⋮& \ddots & ⋮\\ S_{n1}& \cdots & S_{nn}\end{array}\right]\text{and}L=\left[\begin{array}{c}{L}_1\\ ⋮\\ L_n\end{array}\right].$$

$$c(x,\hat{z})=\sum _{i\in N}\Vert x_i-{\hat{z}}_i{\Vert }^2+\lambda \Vert \bar{x}-\bar{z}{\Vert }^2,$$

$$S_{ij}=\bigl{(}\delta_{ij}+\tfrac{\lambda}{n^{2}}\bigr{)}\mathbf{I}.$$

$$c(x,\hat{z})=\sum _{i\in N}\Vert x_i-{\hat{z}}_i{\Vert }^2+\lambda \sum _{i\in N\setminus n}\Vert d_i-{\hat{d}}_i{\Vert }^2,$$

$$S_{ij}=\{\begin{array}{ll}(1+2\lambda )\mathbf{I},& i=j\in \{2,\dots ,n-1\}\\ (1+\lambda )\mathbf{I},& i=j\in \{1,n\}\\ -\lambda \mathbf{I},& j\in \{i+1,i-1\}\\ 0,& \text{otherwise},\end{array}$$

$$c(x,{\hat{z}}_1,\dots ,{\hat{z}}_n)=\sum _{i\in N}\sum _{j\in N}(x-{\hat{z}}_i{)}^{⊺}{S}_{ij}(x-{\hat{z}}_j).$$

$$J(g):=\text{mathds}E[c(x,\hat{z})].$$

$${\hat{z}}_i=L_i{\hat{x}}_0+F_i{\tilde{y}}_i,\forall i\in N,$$

$$\sum_{j\in N}\Big{[}S_{ij}F_{j}\hat{\Sigma}_{ji}-S_{ij}L_{j}\hat{\Theta}_{i}\Big{]}=0,\quad\forall i\in N.$$

$$F={\Gamma }^{-1}\eta ,$$

$$\text{where}F$$

$$\eta$$

$$\Gamma$$

$$J^{\ast }=\mathrm{Tr}(L^{⊺}SL{P}_0)-{\eta }^{⊺}{\Gamma }^{-1}\eta ,$$

$${\Gamma }_{ij}$$

$${\eta }_i$$

$$F={\Gamma }^{-1}\eta =\frac{1}{1+\alpha {\sigma }^2}\left[\begin{array}{c}1\\ 1\end{array}\right],$$

$$J^{*}=\Bigl{(}\sum_{i,j}S_{ij}\Bigr{)}-\eta^{\mathchoice{\raisebox{0.0pt}{$\displaystyle\intercal$}}{\raisebox{0.0pt}{$\textstyle\intercal$}}{\raisebox{0.0pt}{$\scriptstyle\intercal$}}{\raisebox{0.0pt}{$\scriptscriptstyle\intercal$}}}F=(2+\lambda)\frac{\alpha\sigma^{2}}{1+\alpha\sigma^{2}}.$$

$$x(t+1)=Ax(t)+w(t),$$

$$y_i(t)=C_{i}x(t)+v_i(t),$$

$$y(t)=Cx(t)+v(t),$$

Full text

I Introduction

Emerging applications in autonomous vehicles and decentralized control of UAVs (unmanned aerial vehicles) give rise to estimation problems where multiple agents use local measurements to estimate the state of the shared environment in which they are operating and then use these estimates to act in the environment. In the resulting decentralized estimation problems, the objective is to minimize the weighted mean-square error between the true state and the decentralized estimates generated by all agents. We call such a coupled mean-square error as team mean-squared error and the resulting estimates as minimum team mean-squared error (MTMSE) estimates.

For example, consider a platoon of self-driving vehicles where the estimation objective is to ensure that the position estimates of each vehicle are close to the true position of the vehicle and, at the same time, the difference between the position estimates of adjacent vehicles are close to the true difference between the positions. Or consider a fleet of UAVs (unmanned aerial vehicles) where the estimation objective is to ensure that the position estimates of each UAV are close to the true position of the UAV and, at the same time, the centroid of the estimates of all UAVs is close to the true centroid of their positions. A salient feature of these examples is that there are multiple agents who generate state estimates based on different information and the objective is to minimize a weighted mean-squared error between the true state and the decentralized estimates generated by all agents.

We first start with a simple example to illustrate that MTMSE estimates are different from the standard MMSE (minimum mean-squared error) estimates. Consider a system with two agents, indexed by $i\in\{1,2\}$, which observe the state of nature $x\sim\mathcal{N}(0,1)$ with noise. In particular, the measurement $y_{i}\in\mathds{R}$ of agent $i$ is

[TABLE]

where $x$, $v_{1}$, and $v_{2}$ are independent.

Agent $i\in\{1,2\}$ generates an estimate $\hat{z}_{i}=g_{i}(y_{i})\in\mathds{R}$ based on its local measurements, where $(g_{1},g_{2})$ is any arbitrary estimation strategy. The objective is to ensure that $\hat{z}_{i}$ is close to $x$ and at the same time the average $(\hat{z}_{1}+\hat{z}_{2})/2$ of the estimates is close to $x$. Thus, the estimation error $J(g_{1},g_{2})$ of the estimation strategy $(g_{1},g_{2})$ is given by

[TABLE]

where $\lambda\in\mathds{R}_{>0}$. Naively choosing $\hat{z}_{i}$ as the MMSE estimate of $x$ given $y_{i}$, i.e., choosing

[TABLE]

gives an estimation error of

[TABLE]

This naive strategy does not minimize the team mean-squared error given by (1), even within the class of linear estimation strategies. To see this, we identify the best linear estimation strategy. Let

[TABLE]

where $F$ is same for both agents due to symmetry. The estimation error for this linear strategy is

[TABLE]

which is convex in $F$. The value of gain $F$ which minimizes this estimation error is

[TABLE]

where $\alpha=(1+\lambda/4)/(1+\lambda/2)$. The corresponding estimation error is

[TABLE]

Note that for large $\lambda$, $\alpha\approx 1/2$ and the relative improvement

[TABLE]

is significant for moderate values of $\sigma$. For example, for ${\sigma=1}$, the relative percentage improvement is 12.5%. A plot of the relative percentage improvement $\Delta$ as a function of the variance $\sigma$ for different values of $\lambda$ is shown in Fig. 1.

The relative percentage improvement $\Delta\coloneqq(J^{\mathrm{mmse}}-J^{\mathrm{lin}})/{J^{\mathrm{lin}}}\times 100$ as a function of $\sigma$ for different values of $\lambda$ is shown in Fig. 1. The improvement is significant for higher values of $\lambda$.

This significant improvement over MMSE estimates for a simple example motivates the central question of this paper: what are the estimation and filtering strategies that minimize the team mean-squared error? We start by modeling and answering this question for estimation in Sec. II. Then, we model and answer this question for filtering, where we assume that agents are connected over a graph and can share their measurements over a communication graph in Sec. III. We generalize the filtering results to infinite horizon setup in Sec. III-F. Finally, we present examples to illustrate that MTMSE estimates significantly outperform MMSE and consensus Kalman filtering estimates.

I-A Literature overview

Following the seminal work of Kalman [2] on recursive MMSE filtering, several variations of single- and multi-agent MMSE filtering have been investigated in the literature. However, as far as we are aware, there are only two references which have investigated estimation or filtering for the MTMSE objective [3, 4]. Both references investigated multi-agent filtering of a continuous time linear stochastic process. In [3], each agent observes a noise corrupted measurement of the state and the objective is to minimize a specific form of team mean-squared error. The key idea of [3] is to consider an augmented state and observation model and formulate the team mean-square error as the squared norm of an appropriately defined inner product of these augmented variables. It is shown that team mean-squared filtering problem can be formulated as a Hilbert space mean-squared error filtering problem and, therefore, solved using an appropriate Kalman filter. The model considered in [4] is similar except that each agent has multiple observation channels and, at each time, can select which observation channel to use. The solution approach is similar to [3].

Although [3, 4] are able to transform a MTMSE filtering problem to a Hilbert space MMSE filtering problem, the approach has several limitations. First, and most importantly, the approach of [3, 4] is only applicable to a specific form of MTMSE cost. The formulation of the team mean-squared error as a squared norm of an appropriately defined inner product does not hold for the more general team mean-squared error considered in this paper. In particular, the form of the team mean-squared error considered in the practical examples in Sec. IV cannot be written as the squared norm of an appropriate inner product. Second, the size augmented state variables used in [3, 4] scales linearly with the number of agents. In particular, for a $n$-agent MTMSE filtering where the state is of dimension $d_{x}$, the augmented state (and therefore the augmented estimate) is of dimension $n(d_{x})^{2}\times nd_{x}$. Thus the resulting Kalman filter needs to keep track of $n^{2}(d_{x})^{3}\times n^{2}(d_{x})^{3}$ dimensional covariance matrix. In contrast, the solution that we propose only requires a Kalman filter with a $d_{x}\times d_{x}$ dimensional error covariance. Finally, [3, 4] did not consider sharing of measurements among the agents. Such a sharing of measurements is a key feature of the general filtering model that we consider in this paper.

Estimation problems with coupling between the estimates have been considered in the economics literature [5, 6, 7]. However, in such models, agents are strategic and want to minimize an individual estimation objective. The solution concept is identifying estimation strategies which are in Nash equilibrium which is different from the solution concept of minimizing a common team estimation error considered here.

There is a rich literature on multi-agent filtering for distributed sensor fusion [8, 9, 10, 11, 12] as well as for distributed simultaneous localization and mapping (SLAM) in robotics [13, 14, 15]. There is also a rich literature on multi-agent estimation using consensus and gossip Kalman filters [16, 17, 18, 19, 20, 21] (and references therein). However, all these methods only consider MMSE filtering. As illustrated by motivating example presented at the beginning, MTMSE estimates can be significantly different from MMSE estimates. So, the vast literature on multi-agent MMSE filtering is not directly applicable for MTMSE filtering.

I-B Contributions of the paper

The salient feature of the model is that agents are informationally decentralized and need to cooperate to minimize a common team estimation objective. Our focus is to identify the structure of estimation strategies that find MTMSE when the graph topology, system dynamics, and the noise covariances are known to all agents.

We consider the problem of minimizing the team mean-squared error in an estimation problem where the measurements of the agents may be split into a common measurement and local measurements.111If no such split is possible, then the common measurement is simply empty. Using tools from team theory [22], we show that the optimal MTMSE estimate is a sum of two terms. The first term is the MMSE estimate of the state given the common measurement. The second term is a linear function of the innovation in the local measurement given the common measurement. Furthermore, the corresponding gains are computed by solving a system of matrix equation, which can be converted into a linear system of equations using vectorization.

We then consider the problem of minimizing the sum of team mean-squared errors over time in a filtering problem where the agents share their measurements with their neighbors over a completely connected communication graph. Since the graph is completely connected, the information available at each agent can be split into common information and local information. We show that the structure of the optimal MTMSE estimates identified in the estimation setup continue to hold for filtering as well. We setup an appropriate linear system with delayed observation to derive recursive formulas for the MMSE estimate of the state based on the common information and the innovation in the local measurements given the common measurements. We also derive recursive formulas for computing various covariances needed to compute the gain which multiplies the innovation term in the optimal estimates.

Finally, we show that under standard stabilizability and detectability conditions, a time-homogeneous estimation strategy is optimal for minimizing the long-term average team mean-squared error.

A preliminary version of this paper appeared in [1], where the main result for the filtering problem (Theorem 2) was stated. The proof of Theorem 2 relies heavily on the results for the estimation problem (Theorem 1) which was not included in [1]. Neither were the generalization to infinite horizon (Theorem 3). The detailed numerical experiments and the comparison with MMSE estimate and consensus Kalman filtering (Section IV), the detailed comparison with [3, 4] (Section I), the relation between the MTMSE estimates and decentralized control (Section V-B), and the trade-off between MTMSE filter complexity and estimation accuracy (Section V-C) are new as well.

I-C Notation

Let $\delta_{ij}$ denote the Kronecker delta function (which is one if $i=j$ and zero otherwise). Given a matrix $A$, $A_{ij}$ denotes its $(i,j)$-th element, $A_{i\bullet}$ denotes its $i$-th row, $A_{\bullet j}$ denotes its $j$-th column, A^{\mathchoice{\raisebox{0.0pt}{\displaystyle\intercal}}{\raisebox{0.0pt}{\textstyle\intercal}}{\raisebox{0.0pt}{\scriptstyle\intercal}}{\raisebox{0.0pt}{\scriptscriptstyle\intercal}}} denotes its transpose, $\operatorname{vec}(A)$ denotes the column vector of $A$ formed by vertically stacking the columns of $A$. Given a vector $x$, $\|x\|^{2}$ denotes x^{\mathchoice{\raisebox{0.0pt}{\displaystyle\intercal}}{\raisebox{0.0pt}{\textstyle\intercal}}{\raisebox{0.0pt}{\scriptstyle\intercal}}{\raisebox{0.0pt}{\scriptscriptstyle\intercal}}}x. Given matrices $A$ and $B$, $\mathrm{diag}(A,B)$ denotes the matrix obtained by putting $A$ and $B$ in diagonal blocks, and $A\otimes B$ denotes the Kronecker product of the two matrices. Given matrices $A$ and $B$ with the same number of columns, $\operatorname{rows}(A,B)$ denotes the matrix obtained by stacking $A$ on top of $B$. Given a squared matrix $A$, $\operatorname{Tr}(A)$ denotes the sum of its diagonal elements. Given a symmetric matrix $A$, the notation $A>0$ and $A\geq 0$ mean that $A$ is positive definite and semi-definite, respectively. $\textbf{1}_{n\times m}$ is a $n\times m$ matrix with all elements being equal to one. $\textbf{0}_{n}$ is a square $n\times n$ matrix with all elements being equal to zero. $\mathbf{I}_{n}$ is the $n\times n$ identity matrix. We omit the subscript from $\mathbf{I}_{n}$ when the dimension is clear from context. We sometimes consider random vectors $X=(x_{1},\dots,x_{k})$ as a set with random elements $\{x_{1},\dots,x_{k}\}$. In particular, given two random vectors $X=(x_{1},\dots,x_{k})$ and $Y=(y_{1},\dots,y_{m})$, we define $X\bigcap Y$ to mean $\operatorname{vec}(\{x_{1},\dots,x_{k}\}\bigcap\{y_{1},\dots,y_{m}\})$. Similarly, we use $X\setminus Y$ to mean $\operatorname{vec}(\{x_{1},\dots,x_{k}\}\setminus\{y_{1},\dots,y_{m}\})$.

Given any vector valued process $\{y(t)\}_{t\geq 1}$ and any time instances $t_{1}$, $t_{2}$ such that $t_{1}\leq t_{2}$, $y(t_{1}{\mathbin{:}}t_{2})$ is a short hand notation for $\operatorname{vec}(y(t_{1}),y(t_{1}+1),\dots,y(t_{2}))$. Given matrices $\{A(i)\}_{i=1}^{n}$ with the same number of rows and vectors $\{w(i)\}_{i=1}^{n}$, $\operatorname{rows}(\bigodot_{i=1}^{n}A(i))$ and $\operatorname{vec}(\bigodot_{i=1}^{n}w(i))$ denote $\operatorname{rows}(A(1),\dots,A(n))$ and $\operatorname{vec}(w(1),\dots,w(n))$, respectively.

Given random vectors $x$ and $y$, $\mathds{E}[x]$ and $\operatorname{var}(x)$ denote the mean and variance of $x$ while $\operatorname{cov}(x,y)$ denotes the covariance between $x$ and $y$.

II Minimum team mean-squared error (MTMSE) estimation

II-A Model and problem formulation

Consider a system with $n$ agents that are indexed by the set $N=\{1,\dots,n\}$. The agents are interested in estimating the state $x\in\mathds{R}^{d_{x}}$ of nature. Agent $i$ makes a local measurement $y_{i}\in\mathds{R}^{d^{i}_{y}}$, $i\in N$. In addition, all agents observe a common measurement, which we denote by $y_{0}\in\mathds{R}^{d^{0}_{y}}$. We use $N_{0}$ to denote the set $\{0,1,\dots,n\}$.

The variables $(x,y_{0},y_{1},\dots,y_{n})$ are assumed to be jointly Gaussian zero-mean random variables. For any $i,j\in N_{0}$, let $\Theta_{i}=\operatorname{cov}(x,y_{i})$ and $\Sigma_{ij}=\operatorname{cov}(y_{i},y_{j}).$

Agent $i\in N$ generates an estimate $\hat{z}_{i}\in\mathds{R}^{d_{z}^{i}}$ according to an estimation rule $g_{i}$, i.e., $\hat{z}_{i}=g_{i}(y_{0},y_{i})$. Given weight matrices $\{S_{ij}\}_{i,j\in N}$ and $\{L_{i}\}_{i\in N}$, where $S_{ij}\in\mathds{R}^{d_{z}^{i}\times d_{z}^{j}}$ and $L_{i}\in\mathds{R}^{d_{z}^{i}\times d_{x}}$, the performance is measured by the team estimation error given by:

[TABLE]

Let $\hat{z}=\operatorname{vec}(\hat{z}_{1},\dots,\hat{z}_{n})$ denote the estimate of all agents. The team estimation error $c(x,\hat{z})$ is a weighted quadratic function of $(Lx-\hat{z})$. In particular,

[TABLE]

where $S$ and $L$ are given by

[TABLE]

We assume that the matrix $S$ is positive definite.

We now present a few examples of the estimation error function of the form (3):

Suppose $x=\operatorname{vec}(x_{1},\dots,x_{n})$, where $x_{i}$ is the local state of agent $i\in N$. Suppose the agents want to estimate their own local state, but at the same time, want to make sure that the average $\bar{z}\coloneqq\frac{1}{n}\sum_{i\in N}\hat{z}_{i}$ of their estimates is close to the average $\bar{x}\coloneqq\frac{1}{n}\sum_{i\in N}x_{i}$ of their local states. In this case, the team mean-squared error function is

[TABLE]

where $\lambda\in\mathds{R}_{>0}$. This can be written in the form (3) with $L=\mathbf{I}$, and

[TABLE] 2. 2.

Suppose the agents are moving in a line (e.g., a vehicular platoon) or in a closed shape (e.g., UAVs flying in a formation) and want to estimate their local state but, at the same time, want to ensure that the difference $\hat{d}_{i}\coloneqq\hat{z}_{i}-\hat{z}_{i+1}$ between their estimates is close to the difference $d_{i}\coloneqq x_{i}-x_{i+1}$ of their local states.

For example when agents are moving in a line, the team mean-squared error function is

[TABLE]

where $\lambda\in\mathds{R}_{>0}$. This can be written in the form (3) with $L=\mathbf{I}$ and

[TABLE]

A similar weight matrix can be obtained for the case when agents are moving in a closed shape. 3. 3.

Suppose each agent generates an estimate $\hat{z}_{i}\in\mathds{R}^{d_{x}}$ of the state $x$ of nature and the objective is to minimize

[TABLE]

This can be written in the form (3) with $L=\mathbf{1}_{n\times 1}\otimes\mathbf{I}_{d_{x}\times d_{x}}$. This cost function is equivalent to the team mean-squared error considered in [3, 4].

We are interested in the following optimization problem.

Problem 1

Given the covariance matrices $\{\Theta_{i}\}_{i\in N_{0}}$ and $\{\Sigma_{ij}\}_{i,j\in N_{0}}$ and weight matrices $L$ and $S$, choose the estimation strategy $g=(g_{1},\dots,g_{n})$ to minimize the expected team estimation error $J(g)$ given by

[TABLE]

Remark 1

In Problem 1, the system model is common knowledge among all agents. Thus, it may be viewed as a problem of “centralized planning and decentralized execution.” The key conceptual difficulty in the problem is that the estimates are generated using different information (recall that the information available at agent $i$ is $(y_{0},y_{i})$) with the objective of minimizing a common coupled team estimation error given by (3). This feature makes the Problem 1 conceptually different from the standard estimation problem of minimizing the MMSE error. □

II-B Optimal team estimation strategy

We define three auxiliary variables:

• •

All agents’ common estimate of state $x$ given the common measurement $y_{0}$ at all agents. We denote this estimate by $\hat{x}_{0}$ and it is equal to $\mathds{E}[x|y_{0}]$.

• •

All agents’ common estimate of agent $i$’s measurement $y_{i}$ given the common measurement $y_{0}$. We denote this estimate by $\hat{y}_{i}$ and it is equal to $\mathds{E}[y_{i}|y_{0}]$.

• •

The innovation in the local measurement of agent $i$ with respect to the common measurement. We denote this innovation $\tilde{y}_{i}$ and it is equal to $y_{i}-\hat{y}_{i}$.

Let $\hat{\Theta}_{i}$ denote the covariance $\operatorname{cov}(x,\tilde{y}_{i})$ and $\hat{\Sigma}_{ij}$ denote the covariance $\operatorname{cov}(\tilde{y}_{i},\tilde{y}_{j})$. From elementary properties of Gaussian random variables, we have the following:

Lemma 1

The covariance matrices defined above are given by

$\hat{\Theta}_{i}=\Theta_{i}-\Theta_{0}\Sigma_{00}^{-1}\Sigma_{0i}$. 2. 2.

$\hat{\Sigma}_{ij}=\Sigma_{ij}-\Sigma_{i0}\Sigma_{00}^{-1}\Sigma_{0j}$.

Therefore, the auxiliary variables defined above are given by

$\hat{x}_{0}=\Theta_{0}\Sigma_{00}^{-1}y_{0}$. 2. 4.

$\hat{y}_{i}=\Sigma_{ij}\Sigma_{00}^{-1}y_{0}$.

Furthermore, we have

$\mathds{E}[x_{i}|y_{0},y_{i}]=\hat{x}_{0}+\hat{\Theta}_{i}\hat{\Sigma}_{ii}^{-1}\tilde{y}_{i}$*. * 2. 6.

$\mathds{E}[\tilde{y}_{j}\,|\,y_{0},y_{i}]=\hat{\Sigma}_{ji}\hat{\Sigma}_{ii}^{-1}\tilde{y}_{i}$.

□

The result follows from elementary properties of Gaussian random variables. Then, we have the following.

Theorem 1

The estimation strategy that minimizes the team mean-squared error in Problem 1 is a linear function of the measurements. Specifically, the MTMSE estimate may be written as

[TABLE]

where the gains $\{F_{i}\}_{i\in N}$ satisfy the following system of matrix equations:

[TABLE]

If $\hat{\Sigma}_{ii}>0$ for all $i\in N$, then (9) has a unique solution which can be written as

[TABLE]

[TABLE]

Furthermore, the minimum team mean-squared error is given by

[TABLE]

where $S_{i}=[S_{i1},\dots,S_{in}]$ and $P_{0}=\operatorname{var}(x-\hat{x}_{0}).$ □

The proof of Theorem 1 is presented in Appendix A.

To illustrate this result, consider the two agent example presented in the introduction. In that model, there is no common measurement. So $\hat{x}_{0}=0$, $\hat{y}_{i}=0$, and therefore $\tilde{y}_{i}=y_{i}$. Moreover, $\hat{\Sigma}_{ij}=1+\sigma^{2}\delta_{ij}$ and $\hat{\Theta}_{i}=1$. Therefore,

[TABLE]

Thus, the optimal gains are

[TABLE]

where $\alpha=(1+\lambda/4)/(1+\lambda/2)$ and the minimum team mean-squared error is

[TABLE]

Thus, we recover the results obtained by brute force calculations in the introduction.

Remark 2

In (8), the first term of the estimate is the MMSE estimate of the current state given the common measurements. The second term may be viewed as a “correction” which depends on the innovation in the local measurement. A salient feature of the result is that the gains $\{F_{i}\}_{i\in N}$ depend on the weight matrix $S$. □

Remark 3

When $S$ is block diagonal, there is no cost coupling among the agents and Problem 1 reduces to $n$ separate problems. Thus, the MMSE estimates $L_{i}\hat{x}_{i}$ are also the MTMSE estimates. □

III Minimum team mean-squared error (MTMSE) filtering

In this section, we consider the problem of filtering to minimize team mean-squared error when agents share information over a communication graph. We start with a quick overview of graph theoretic terminology.

III-A Overview of graph theoretic terminology

A directed weighted graph $\mathcal{G}$ is an ordered set $(N,E,\tau)$ where $N$ is the set of nodes and $E\subset N\times N$ is the set of ordered edges, and $\tau\colon E\to\mathds{R}^{k}$ is a weight function. An edge $(i,j)$ in $E$ is considered directed from $i$ to $j$; $i$ is the in-neighbor of $j$; $j$ is the out-neighbor of $i$; and $i$ and $j$ are neighbors. The set of in-neighbors of $i$, called the in-neighborhood of $i$, is denoted by $N^{-}_{i}$; the set of out-neighbors of $i$, called the out-neighborhood, is denoted by $N^{+}_{i}$.

In a directed graph, a directed path $(v_{1},v_{2},\dots,v_{k})$ is a weighted sequence of distinct nodes such that $(v_{i},v_{i+1})\in E$. The length of a path is the weighted number of edges in the path. The geodesic distance between two nodes $i$ and $j$, denoted by $\ell_{ij}$, is the shortest weight length of all paths connecting the two nodes. The weighted diameter of the graph is the largest weighted geodesic distance between any two nodes. A directed graph is called strongly connected if for every pair of nodes $i,j\in N$, there is a directed path from $i$ to $j$ and from $j$ to $i$. A directed graph is called complete if for every pair of nodes $i,j\in N$, there is a directed edge from $i$ to $j$ and from $j$ to $i$.

III-B Model and problem formulation

III-B1 Observation Model

Consider a linear stochastic process $\{x(t)\}_{t\geq 1}$, $x(t)\in\mathds{R}^{d_{x}}$, where $x(1)\sim\mathcal{N}(0,\Sigma_{x})$ and for $t\geq 1$,

[TABLE]

where $A$ is a $d_{x}\times d_{x}$ matrix and $w(t)\in\mathds{R}^{d_{x}}$, $w(t)~{}\sim~{}\mathcal{N}(0,Q)$, is the process noise. There are $n$ agents, indexed by $N=\{1,\dots,n\}$, which observe the process with noise. At time $t$, the measurement $y_{i}(t)\in\mathds{R}^{d_{y}^{i}}$ of agent $i\in N$ is given by

[TABLE]

where $C_{i}$ is a $d_{y}^{i}\times d_{x}$ matrix and $v_{i}(t)\in\mathds{R}^{d_{y}^{i}}$, $v_{i}(t)~{}\sim~{}\mathcal{N}(0,R_{i})$, is the measurement noise. Eq. (13) may be written in vector form as

[TABLE]

where $C=\operatorname{rows}(C_{1},\dots,C_{n})$, $y(t)=\operatorname{vec}(y_{1}(t),\dots,y_{n}(t))$, and $v(t)=\operatorname{vec}(v_{1}(t),\dots,v_{n}(t))$.

The agents are connected over a communication graph $\mathcal{G}$, which is a strongly connected weighted directed graph with vertex set $N$. For every edge $(i,j)$, the associated weight $\tau_{ij}$ is a positive integer that denotes the communication delay from node $i$ to node $j$.

Let $I_{i}(t)$ denote the information available to agent $i$ at time $t$. We assume that agent $i$ knows the history of all its measurements and $\tau_{ji}$ step delayed information of its in-neighbor $j$, $j\in N^{-}_{i}$, i.e.,

[TABLE]

In (14), we implicitly assume that $I_{i}(t)=\emptyset$ for any $t\leq 0$.

Let $\zeta_{i}(t)=I_{i}(t)\setminus I_{i}(t-1)$ denote the new information that becomes available to agent $i$ at time $t$. Then, $\zeta_{i}(1)=y_{i}(1)$ and for $t>1$,

[TABLE]

It is assumed that at each time $t$, agent $j\in N$, communicates $\zeta_{j}(t)$ to all its out-neighbors. This information reaches the out-neighbor $i$ of agent $j$ at time $t+\tau_{ji}$.

Some examples of the communication graph are as follows.

Example 1

Consider a complete graph with $\tau$-step delay along each edge. The resulting information structure is

[TABLE]

which is the $\tau$-step delayed sharing information structure [23]. □

Example 2

Consider a strongly connected graph with unit delay along each edge. Let $\tau^{*}=\max_{i,j\in N}\ell_{ij},$ denote the weighted diameter of the graph and $N^{k}_{i}=\{j\in N:\ell_{ji}=k\}$ denote the $k$-hop in-neighbors of $i$ with $N^{0}_{i}=\{i\}$. The resulting information structure is

[TABLE]

which we call the neighborhood sharing information structure. □

At time $t$ agent $i\in N$ generates an estimate $\hat{z}_{i}(t)\in\mathds{R}^{d^{i}_{z}}$ of $L_{i}x(t)$ (where $L_{i}$ is a $\mathds{R}^{d_{z}^{i}\times d_{x}}$ matrix) according to

[TABLE]

where $g_{i,t}$ is a measurable function called the estimation rule at time $t$. The collection $g_{i}\coloneqq(g_{i,1},g_{i,2},\dots)$ is called the estimation strategy of agent $i$ and $g\coloneqq(g_{1},\dots,g_{n})$ is the team estimation strategy profile of all agents.

III-B2 Estimation Cost

Let $\hat{z}(t)=\operatorname{vec}(\hat{z}_{1}(t),\dots,\hat{z}_{n}(t))$ denote the estimate of all agents. As in Sec. II, we assume that the estimation error $c(x(t),\hat{z}(t))$ is a weighted quadratic function of $(Lx(t)-\hat{z}(t))$ of the form

[TABLE]

Examples of such estimation error functions were given in Sec. II-A.

III-B3 Problem Formulation

It is assumed that the system satisfies the following assumptions.

1. (A1)

The cost matrix $S$ is positive definite. 2. (A2)

The noise covariance matrices $\{R_{i}\}_{i\in N}$ are positive definite and $Q$ and $\Sigma_{x}$ are positive semi-definite. 3. (A3)

The primitive random variables $(x(1),\allowbreak\{w(t)\}_{t\geq 1},\allowbreak\{v_{1}(t)\}_{t\geq 1},\allowbreak\dots,\{v_{n}(t)\}_{t\geq 1})$ are independent. 4. (A4)

For any square root $D$ of matrix $Q$ such that $DD=Q$, $(A,D)$ is stabilizable. 5. (A5)

$(A,C)$ is detectable.

We are interested in the following optimization problem.

Problem 2** (Finite Horizon)**

Given matrices $A$, $\{C_{i}\}_{i\in N}$, $\Sigma_{x}$, $Q$, $\{R_{i}\}_{i\in N}$, $L$, $S$, a communication graph $\mathcal{G}$ (and the corresponding weights $\tau_{ij}$), and a horizon $T$, choose a team estimation strategy profile $g$ to minimize $J_{T}(g)$ given by

[TABLE]

Problem 3** (Infinite Horizon)**

Given matrices $A$, $\{C_{i}\}_{i\in N}$, $\Sigma_{x}$, $Q$, $\{R_{i}\}_{i\in N}$, and a communication graph $\mathcal{G}$ (and the corresponding weights $\tau_{ij}$), choose a team estimation strategy profile $g$ to minimize $\bar{J}(g)$ given by

[TABLE]

As was the case for the estimation problem presented in Sec. II, a salient feature of the model is that the estimates are generated using different information while the objective is to minimize a common coupled estimation error given by (16) or (17). This feature makes the Problems 2 and 3 conceptually different from the standard filtering problem of minimizing the MMSE error.

Remark 4

For Problem 2, the assumption that the dynamics, measurements, and cost are time-homogeneous is made simply for convenience of notation. As will be evident from the analysis, the results for Problem 2 generalize to the setting of time-varying dynamics, measurements, and cost in a natural manner. □

III-C Roadmap of the results

The main idea behind identifying a solution for Problem 2 is as follows. We observe that the choice of the estimates only affects the instantaneous estimation error but does not affect the evolution of the system or the estimation error in the future. Therefore, the problem of choosing an estimation profile $g=(g_{1},\dots,g_{n})$ to minimize $J_{T}(g)$ is equivalent to solving the following $T$ separate optimization problems:

[TABLE]

Since the communication graph is strongly connected, the information $I_{i}(t)$ available at agent $i$ can be written as $I^{\mathrm{com}}(t)\cup I^{\mathrm{loc}}_{i}(t)$, where

[TABLE]

is the common information among all agents (recall that $\tau^{*}$ is the weighted diameter of the communication graph) and

[TABLE]

is the location information at agent $i$. Thus, we may view Problem (18) as an estimation problem with $n$ agents where agents have local and common information and, therefore, use the results of Sec. II to derive the MTMSE filtering strategy. To do so, we define variables which are equivalent to the auxiliary variables defined in Sec. II-B:

• •

All agents’ common estimate of state $x(t)$ given the common information $I^{\mathrm{com}}(t)$ at all agents. We denote this estimate by $\hat{x}^{\mathrm{com}}(t)$ and it is equal to $\mathds{E}[x(t)|I^{\mathrm{com}}(t)]$.

• •

All agents’ common estimate of the local information at agent $i$ given the common information. We denote this estimate by $\hat{I}^{\mathrm{loc}}_{i}(t)$ and it is equal to $\mathds{E}[I^{\mathrm{loc}}_{i}(t)|I^{\mathrm{com}}(t)]$.

• •

The innovation in the local information at agent $i$ with respect to the common information. We denote this innovation by $\tilde{I}_{i}(t)$ and it is equal to $I_{i}(t)-\hat{I}_{i}(t)$.

Furthermore, we let $\hat{\Theta}_{i}(t)$ denote the covariance $\operatorname{cov}(x(t),\tilde{I}_{i}(t))$ and $\hat{\Sigma}_{ij}(t)$ denote the covariance $\operatorname{cov}(\tilde{I}^{\mathrm{loc}}_{i}(t),\tilde{I}^{\mathrm{loc}}_{j}(t))$.

In order to use the results of Theorem 1, we need to derive expressions for recursively updating the above variables and covariances, which we do next.

III-D Recursive expressions for auxiliary variables and covariances

The information structure of the problem is effectively equal to $\tau^{*}$-step delayed information structure [23]. To derive recursive expressions for auxiliary variables and covariances, we follow the central idea of [23] and express the system variables in terms of delayed state $x(t-\tau^{*}+1)$.

III-D1 Delayed state estimates and common estimates

We define

[TABLE]

as the delayed state estimate of the state and let

[TABLE]

denote the corresponding estimation error and $P(t-\tau^{*}+1)=\operatorname{var}(\tilde{x}(t-\tau^{*}+1))$ denote the estimation error covariance. Note that $\hat{x}(t-\tau^{*}+1)$ is the one-step prediction estimate in centralized Kalman filtering and can be updated as follows. Start with $\hat{x}(1)=0$ and for $t\geq 1$, update

[TABLE]

is the Kalman gain. Furthermore, the error covariance $P(t)$ can be pre-computed recursively using the forward Riccati equation: $P(1)=\Sigma_{x}$ and for $t\geq 1$,

[TABLE]

where $\Delta(t)=I-K(t)C$.

Now, observe that we can compute the common estimate $\hat{x}^{\mathrm{com}}(t)$ using a $(\tau^{*}-1)$-step propagation of the delayed state estimate $\hat{x}(t-\tau^{*}+1)$ as follows:

[TABLE]

III-D2 Local estimates and local innovation

To find a convenient expression for local innovation $\tilde{I}^{\mathrm{loc}}_{i}(t)$, we express $I^{\mathrm{loc}}_{i}(t)$ in terms of the delayed state $x(t-\tau^{*}+1)$. For that matter, for any $t,\ell\in\mathbb{Z}_{>0}$, define the $d_{x}\times 1$ random vector $w^{(k)}(\ell,t)$ as follows:

[TABLE]

where $w^{(k)}(\ell,t)$ is the weighted accumulated process noise from time $\max\{1,t-k\}$ to time $t-\ell-1$. Note that $w^{(k)}(\ell,t)=0$ if $t\leq\min\{k,\ell+1\}$ or $\ell\geq k$. For any $t\geq k$, we may write

[TABLE]

By definition $I^{\mathrm{loc}}_{i}(t)\subseteq y(t-\tau^{*}+1{\mathbin{:}}t)$. Thus, for any $i\in N$, we can identify matrix $C^{\mathrm{loc}}_{i}$ and random vectors $w^{\mathrm{loc}}_{i}(t)$ and $v^{\mathrm{loc}}_{i}(t)$ (which are linear functions of $w(t-\tau^{*}+1{\mathbin{:}}t-1)$ and $v_{i}(t-\tau^{*}+1{\mathbin{:}}t)$) such that

[TABLE]

As an example, we write the expressions for $(C_{i}^{\mathrm{loc}},w^{\mathrm{loc}}_{i}(t),v^{\mathrm{loc}}_{i}(t))$ for the delayed sharing and neighborhood sharing information structures below. For any $\ell\leq\tau^{*}$, define

[TABLE]

Example 1 (cont.)

For the $\tau$-step delayed sharing information structure $I^{\mathrm{loc}}_{i}(t)=y_{i}(t-\tau+1{\mathbin{:}}t)$. Thus, $C_{i}^{\mathrm{loc}}=\mathcal{C}_{i}(0)$, $w_{i}^{\mathrm{loc}}(t)=\mathcal{W}_{i}(0,t)$, and $v_{i}^{\mathrm{loc}}(t)=\mathcal{V}_{i}(0,t)$. □

Example 2 (cont.)

For the neighborhood sharing information structure, $I_{i}(t)=\bigcup_{k=0}^{\tau^{*}}\bigcup_{j\in N^{k}_{i}}\{y_{j}(1{\mathbin{:}}t-k)\}.$ Thus,

[TABLE]

□

Now, a key-result is the following.

Lemma 2

$w^{\mathrm{loc}}_{i}(t)$, $v^{\mathrm{loc}}_{i}(t)$, $\tilde{x}(t-\tau^{*}+1)$, and $I^{\mathrm{com}}(t)$ are independent. □

Proof

Observe that $I^{\mathrm{com}}(t)=y(1{\mathbin{:}}t-\tau^{*})$ and ${\tilde{x}(t-\tau^{*}+1)}$ are functions of the primitive random variables up to time $t-\tau^{*}$, while $w^{\mathrm{loc}}_{i}(t)$ and $v^{\mathrm{loc}}_{i}(t)$ are functions of the primitive random variables from time ${t-\tau^{*}+1}$ onwards. Thus, $w^{\mathrm{loc}}_{i}(t)$ and $v^{\mathrm{loc}}_{i}(t)$ are independent of $\tilde{x}(t-\tau^{*}+1)$ and $I^{\mathrm{com}}(t)$. Furthermore, (A3) implies that $w^{\mathrm{loc}}_{i}(t)$ and $v^{\mathrm{loc}}_{i}(t)$ are independent of each other. Note that $\tilde{x}(t-\tau^{*}+1)$ is the estimation error when estimating $x(t-\tau^{*}+1)$ given $I^{\mathrm{com}}(t)$ and is, therefore, uncorrelated with $I^{\mathrm{com}}(t)$. Since all random variables are Gaussian, $\tilde{x}(t-\tau^{*}+1)$ and $I^{\mathrm{com}}(t)$ being uncorrelated also means that they are independent. ■

Combining Lemma 2 with (27), we get

[TABLE]

Combining this with (27), we get,

[TABLE]

III-D3 Covariances

Let $P^{w}_{ij}(t)$ denote $\operatorname{cov}(w^{\mathrm{loc}}_{i}(t),w^{\mathrm{loc}}_{j}(t))$ and $P^{v}_{ij}(t)$ denote $\operatorname{cov}(v^{\mathrm{loc}}_{i}(t),v^{\mathrm{loc}}_{j}(t))$. Note that these can be computed from he expressions of $w^{\mathrm{loc}}_{i}(t)$ and $v^{\mathrm{loc}}_{i}(t)$, which were derived earlier based on the communication graph.

Eq. (29) and Lemma 2 imply that

[TABLE]

where $P(t)$ is computed using (22).

Furthermore, Eqs. (25) and (29) and Lemma 2 imply that

[TABLE]

where $P^{\sigma}_{i}(t)=\operatorname{cov}(w^{(\tau^{*}-1)}(0,t),w^{\mathrm{loc}}_{i}(t))$ and $P(t)$ is computed using (22).

III-E Main result for Problem 2

As mentioned in Sec. III-C, the problem of choosing the MTMSE estimation strategy $g=(g_{1},\dots,g_{T})$ to minimize $J_{T}(g)$ is equivalent to solving $T$ separate estimation sub-problems given by (18). Based on Theorem 1, the MTMSE estimate of each of these sub-problems is given as follows.

Theorem 2

Under assumptions (A1)–(A3), the filtering strategy which minimizes the team mean-squared error in Problem 2 is a linear function of the measurements. Specifically, the MTMSE estimates at time $t$ may be written as

[TABLE]

where $\hat{x}^{\mathrm{com}}(t)$ and $\tilde{I}^{\mathrm{loc}}_{i}(t)$ are computed using (23) and (29). The gains $\{F_{i}(t)\}_{i\in N}$ satisfy the following system of matrix equations

[TABLE]

where $\hat{\Sigma}_{ij}(t)$ and $\hat{\Theta}_{i}(t)$ are computed using (30) and (31). Eq. (33) has a unique solution which can be written as

[TABLE]

where

[TABLE]

Furthermore, the minimum team mean-squared error is given by

[TABLE]

where $P_{0}(t)=\operatorname{var}({x(t)-\hat{x}^{\mathrm{com}}(t)})$ and is given by

[TABLE]

and $\Sigma^{w}(t)=\operatorname{var}(w^{(\tau^{*}-1)}(0,t))$. □

Proof

The expressions for the MTMSE estimates (32) and the corresponding gains (33) follow immediately from Theorem 1. Now, since $R_{ii}$ is positive definite (which is part of (A2)), standard results from Kalman filtering [24, Section 3.4] imply that $P(t)$ is positive definite. Using this fact in (30) implies that $\hat{\Sigma}_{ii}(t)$ is positive definite. Therefore, the vectorized formula (34) follows from Lemma 5.

The expression for the minimum team mean-squared error follow from an argument similar to that in the proof of Theorem 1. The expression for $P_{0}(t)$ follows from (23) and (25). ■

Remark 5

Remark 2 about the structure of the MTMSE estimates continues to hold for filtering setup as well. The first term in the MTMSE estimate (32) is the MMSE estimate of the current state based on the common information. The second term is a “correction” which depends on the innovation in the local measurements. □

Remark 6

As in the estimation setup, the gains which multiply the innovation in (32) are coupled and depend on the weight matrix $S$. □

Remark 7

Since we have assumed that the dynamics are time-homogeneous, the processes $\{w^{(\tau^{*}-1)}(0,t)\}_{t\geq\tau^{*}}$, $\{w^{\mathrm{loc}}_{i}(t)\}_{t\geq\tau^{*}}$, and $\{v^{\mathrm{loc}}_{i}(t)\}_{t\geq\tau*}$ are stationary. Hence, for $t\geq\tau^{*}$, the covariance matrices $\Sigma^{w}(t)$, $P^{\sigma}_{i}(t)$, $P^{w}_{ij}(t)$, and $P^{v}_{ij}(t)$ are constant. □

Remark 8

Note that $\hat{\Sigma}_{ij}\otimes S_{ij}=\mathbf{0}$ when $S_{ij}=0$. Therefore, when the weight matrix $S$ is sparse, as is the case for the cost (6), $\hat{\Sigma}_{ij}$ (and, therefore, $P^{w}_{ij}(t)$ and $P^{v}_{ij}(t)$) need to computed only for those $i,j\in N$ for which $S_{ij}\neq\mathbf{0}$. □

III-F Main result for Problem 3

Now, we consider the infinite horizon MTMSE filtering introduced in Problem 3, which can be thought of as a “steady-state” version of Sec. III-E. We first state a standard result from centralized Kalman filtering [24].

Lemma 3

Under (A2)–(A5), for any initial covariance $\Sigma_{x}\geq 0$, the sequence $\{P(t)\}_{t\geq 1}$ given by (21) is weakly increasing and bounded (in the sense of positive semi-definiteness). Thus it has a limit, which we denote by $\bar{P}$. Furthermore,

$\bar{P}$* does not depend on $\Sigma_{x}$.* 2. 2.

$\bar{P}$* is positive semi-definite.* 3. 3.

$\bar{P}$* is the unique solution to the following algebraic Riccati equation.*

[TABLE]

where \bar{K}=\bar{P}C^{\mathchoice{\raisebox{0.0pt}{\displaystyle\intercal}}{\raisebox{0.0pt}{\textstyle\intercal}}{\raisebox{0.0pt}{\scriptstyle\intercal}}{\raisebox{0.0pt}{\scriptscriptstyle\intercal}}}\big{[}C\bar{P}C^{\mathchoice{\raisebox{0.0pt}{\displaystyle\intercal}}{\raisebox{0.0pt}{\textstyle\intercal}}{\raisebox{0.0pt}{\scriptstyle\intercal}}{\raisebox{0.0pt}{\scriptscriptstyle\intercal}}}+R\big{]}^{-1} and $\Delta=I-\bar{K}C$. 4. 4.

The matrix $(A-\bar{K}C)$ is asymptotically stable.

□

Recall from Remark 7 that $\Sigma^{w}(t)$, $P^{\sigma}_{i}(t),P^{w}_{ij}(t)$ and $P^{v}_{ij}(t)$ are constants for $t\geq\tau^{*}$. We denote the corresponding values for $t\geq\tau^{*}$ as $\bar{\Sigma}^{w}$, $\bar{P}^{\sigma}_{i},\bar{P}^{w}_{ij}$, and $\bar{P}^{v}_{ij}$. Now define:

[TABLE]

Lemma 4

Under (A2)–(A5), we have the following:

$\lim_{t\rightarrow\infty}P_{0}(t)=\bar{P}_{0}$. 2. 2.

$\lim_{t\rightarrow\infty}\hat{\Sigma}_{ij}(t)=\bar{\Sigma}_{ij}$. 3. 3.

$\lim_{t\rightarrow\infty}\hat{\Theta}_{i}(t)=\bar{\Theta}_{i}.$**

□

Proof

All relations follow immediately from Lemma 3 and Remark 7. ■

Theorem 3

Under (A1)–(A5), the following time-homogeneous filtering strategy minimizes the team mean-squared error for Problem 3:

[TABLE]

where $\hat{x}^{\mathrm{com}}(t)=A^{\tau^{*}-1}\hat{x}(t-\tau^{*}+1)$ (which is same as (23)), $\hat{x}(t)$ is updated using the steady state version of (20) given by

[TABLE]

and the gains $\{\bar{F}_{i}\}_{i\in N}$ satisfy the following system of matrix equations:

[TABLE]

where $\bar{\Sigma}_{ij}$ and $\bar{\Theta}_{i}$ are given by (39) and (40). Eq. (43) has a unique solution and can be written more compactly as

[TABLE]

where

[TABLE]

Furthermore, the optimal performance is given by

[TABLE]

where $\bar{P}_{0}$ is given by (38). □

The proof of Theorem 3 is presented in Appendix C.

IV Some illustrative examples

In this section, we present a few examples to illustrate the details of the main results.

IV-A Team mean-squared estimation in a UAV formation

Consider a UAV formation with $n$ agents as shown in Fig. 2. Let $N=\{1,\dots,n\}$ and $x_{i}(t)$ denote the state of agent $i\in N$. For the ease of exposition, we assume that $x_{i}(t)\in\mathds{R}$, which could correspond to say the altitude of the UAV. Let $x(t)=\operatorname{vec}(x_{1}(t),\dots,x_{n}(t))$ denote the state of the system, which evolves as

[TABLE]

where $A$ is a known $n\times n$ matrix and $w(t)\sim\mathcal{N}(0,Q)$. The agent $i$ observes the state with noise, i.e.,

[TABLE]

where $v_{i}(t)\sim\mathcal{N}(0,R_{i})$.

The communication graph is as shown in Fig. 2, where each link is assumed to have delay 2. Thus, the information structure is given by

[TABLE]

The objective is to determine the MTMSE filtering for per-step estimation error given by (5), i.e., the agents want to estimate their local state and ensure that the average of the local state estimates is close to the average of their actual states.

We first show the computations of the MTMSE estimates. Observe that $I^{\mathrm{com}}(t)=y(1{\mathbin{:}}t-2)$ and

[TABLE]

Thus, $C^{\mathrm{loc}}_{i}=\operatorname{rows}(C_{i},C_{i}A),$ and

[TABLE]

As argued in Remark 7, the covariance matrices $\Sigma^{w}(t)$, $P^{\sigma}_{i}(t)$, $P^{w}_{ij}(t)$, and $P^{v}_{ij}(t)$ are constant for $t\geq\tau^{*}$. Thus, we only need to compute these for $t=1$ and $t\geq 2$. Note that the weight matrix $S$ is dense, so we do not get the computational savings described in Remark 8.

We have the following:

• •

$\Sigma^{w}(1)=0$ and for $t\geq 2$, $\Sigma^{w}(t)=Q$.

• •

$P^{\sigma}_{i}(1)=\begin{bmatrix}\mathbf{0}_{4\times 1}&\mathbf{0}_{4\times 1}\end{bmatrix}$ and for $t\geq 2$, P^{\sigma}_{i}(t)=\begin{bmatrix}\mathbf{0}_{4\times 1}&QC^{\mathchoice{\raisebox{0.0pt}{\displaystyle\intercal}}{\raisebox{0.0pt}{\textstyle\intercal}}{\raisebox{0.0pt}{\scriptstyle\intercal}}{\raisebox{0.0pt}{\scriptscriptstyle\intercal}}}\end{bmatrix}.

• •

$P^{w}_{ij}(1)=\mathrm{diag}(0,0)$ and for $t\geq 2$, P^{w}_{ij}(t)=\mathrm{diag}(0,C_{i}QC_{j}^{\mathchoice{\raisebox{0.0pt}{\displaystyle\intercal}}{\raisebox{0.0pt}{\textstyle\intercal}}{\raisebox{0.0pt}{\scriptstyle\intercal}}{\raisebox{0.0pt}{\scriptscriptstyle\intercal}}}).

• •

$P^{v}_{ii}(1)=\mathrm{diag}(0,R_{i})$ and $P^{v}_{ii}(t)=\mathrm{diag}(R_{i},R_{i})$.

• •

$P^{v}_{ij}(t)=\mathrm{diag}(0,0)$ for $j\neq i$ and all $t$.

Substituting these, we get that $\hat{\Sigma}_{ij}(1)=\delta_{ij}\mathrm{diag}(0,R_{i})$ and for $t\geq 2$,

[TABLE]

Substituting these in (33) or (34) gives us the optimal gains. The MTMSE estimates can then be computed using (32) as described in Sec. V-A.

We compare the performance of MTMSE filtering strategy with two baselines. The first is MMSE strategy where, each agent ignores the cost coupling and simply generates the MMSE estimates using

[TABLE]

It can be shown that performance of the MMSE strategy is

[TABLE]

Recall that for this particular example we have $L=\mathbf{I}$.

The second is a consensus based Kalman filter as described in [16]. We do not have a closed form expression for the weighted mean square error of the consensus Kalman filter, so we evaluate the performance $J^{\text{CKF}}_{T}$ using Monte Carlo evaluation averaged over $1000$ sample paths.

For the numerical experiments we pick

[TABLE]

$C_{1}=2\times\textbf{1}_{1\times n}$, and for $i\neq 1$, $C_{i}=0.1e_{i}$, where $e_{i}$ is a vector with only the $i_{th}$ element equal to one and the rest zero, $Q=\mathbf{I},R=0.1\mathbf{I}$, and $T=100$.

The relative improvements

[TABLE]

of the MTMSE strategy compared to MMSE strategy and consensus Kalman filtering as a function of $\lambda$ are shown in Fig. 3. These plots show that the MTMSE strategy outperforms the MMSE and consensus Kalman filtering strategies by up to a factor of 4 and 600 in the relative improvements for $n=10$ and $\frac{\lambda}{n^{2}}=10$. This improvement in performance will increase with the number of agents.

IV-B Team mean-squared estimation in a vehicular platoon

Now we consider a vehicular platoon with four agents shown in Fig. 4. As before, let $x_{i}(t)\in\mathds{R}$ denote the position of the platoon. We assume that the dynamics and the observation model are similar to that described in Sec. IV-A (but with different $A$ and $C$ matrices).

The communication graph is as shown in Fig. 4. Thus, the information structure is given by

[TABLE]

The objective is to determine the MTMSE filtering for per-step estimation error given by (6), i.e., the agents want to estimate their local states and ensure that the difference between the estimates of adjacent agents is close to difference between their actual states.

We first show the computations of the MTMSE estimates. Observe that $I^{\mathrm{com}}(t)=y(1{\mathbin{:}}t-3)$ and

[TABLE]

Similar to the previous example, the covariance matrices $\Sigma^{w}(t)$, $P^{\sigma}_{i}(t)$, $P^{w}_{ij}(t)$, and $P^{v}_{ij}(t)$ are constant for $t\geq\tau^{*}$. Thus, we need to compute these for $t=1$, $t=2$, and $t\geq 3$. In addition, since the cost matrix $S$ is sparse, we only need to compute $P^{w}_{ij}(t)$ and $P^{v}_{ij}(t)$ for $j\in\{i-1,i,i+1\}\cap N$ (see Remark 8). The details for computing $\hat{\Sigma}_{ij}$ are similar to the previous section and are omitted due to space limitations. The MTMSE estimates can be computed using (32) as described in Sec. V-A.

We compare the performance of MTMSE filtering strategy with the MMSE strategy and the consensus Kalman filtering as before.

For the numerical experiment in this part, we pick

[TABLE]

$C_{i}=\textbf{I}_{n}$, $Q=\mathbf{I},R=0.1\mathbf{I}$, and $T=100$.

The relative improvements as a function of $\lambda$ are shown in Fig. 5. These plots show that the MTMSE strategy outperforms the MMSE and consensus Kalman filtering strategies by up to a factor of 2 and 800. Again, this improvement in performance will increase with the number of agents.

V Discussion of the results

V-A Implementation of MTMSE filtering strategy

In this section, we provide the details about implementing the MTMSE filtering strategies for both the finite and infinite horizon setups.

V-A1 Implementation of finite horizon MTMSE filtering strategy

Based on Theorem 2, the MTMSE filtering strategy can be implemented as follows.

Computing the gains

The gains $\{F(t)\}_{t=1}^{T}$ are computed offline as follows. First the variance $\{P(t)\}_{t=1}^{T}$ are computed using the forward Riccati equation (22). Then, the covariances $\{\hat{\Sigma}_{ij}(t)\}_{t=1}^{T}$ and $\{\hat{\Theta}_{i}(t)\}_{t=1}^{T}$ are computed for all $i,j\in N$. Thereafter, the gains $\{K(t)\}_{t=1}^{T}$ are computed using (21) and the gains $\{F(t)\}_{t=1}^{T}$ are computed using (34).

Finally, the gains $\{K(t)\}_{t=1}^{T}$ and $\{F_{i}(t)\}_{t=1}^{T}$ are stored in agent $i$.

Computing the MTMSE estimates

Agent $i\in N$ carries out the following computations to generate $\hat{z}_{i}(t)$. First, it computes the delayed centralized estimate $\hat{x}(t-\tau^{*}+1)$ using (20). Then, it uses $\hat{x}(t-\tau^{*}+1)$ to compute $\hat{x}^{\mathrm{com}}(t)$ and $\hat{I}^{\mathrm{loc}}_{i}(t)$ using (23) and (28), respectively. Then, it uses $\hat{x}^{\mathrm{com}}(t)$ and $I^{\mathrm{loc}}_{i}(t)$ to generate the MTMSE estimate as follows

[TABLE]

V-A2 Implementation of infinite horizon MTMSE filtering strategy

Based on Theorem 3, the MTMSE filtering strategy can be implemented as follows.

Computing the gains

The gains $\{\bar{F}_{i}\}$ are computed offline as follows. First the variance $\bar{P}$ is computed using the forward algebraic Riccati equation (37). Then, the covariances $\bar{P}_{0}$, $\bar{\Sigma}_{ij}$, and $\bar{\Theta}_{i}$ are computed for all $i,j\in N$ using (38)-(40). Thereafter, the gain $\bar{K}$ is computed using Lemma 3 and the gain $\bar{F}$ is computed using (44). Finally, the gains $\bar{K}$ and $\bar{F}$ are stored in agent $i$.

Computing the MTMSE estimates

Agent $i\in N$ carries out the following computations to generate $\hat{z}_{i}(t)$. First, it computes the delayed centralized estimate $\hat{x}(t-\tau^{*}+1)$ using (42). Then, it uses $\hat{x}(t-\tau^{*}+1)$ to compute $\hat{x}^{\mathrm{com}}(t)$ and $\hat{I}^{\mathrm{loc}}_{i}(t)$ using (23) and (28), respectively. Then, it uses $\hat{x}^{\mathrm{com}}(t)$ and $I^{\mathrm{loc}}_{i}(t)$ to generate the MTMSE estimate as follows

[TABLE]

V-B Connection to decentralized stochastic control

One of the most celebrated results in centralized stochastic control of linear systems with quadratic cost and Gaussian disturbance (so-called LQG setup) is the separation of estimation and control. In particular, the optimal control action is equal to a gain multiplied by the current state estimate. The computation of the gain matrix and the estimate are separated from each other. The gain matrix is computed based on the solution of a backward Riccati equation where the state estimates are updated based on the Kalman filtering equation (which is a forward Riccati equation). The forward and the backward Riccati equations are decoupled and can be solved separately.

These simplifications do not hold for decentralized control of LQG systems. In general, non-linear strategies may outperform the best linear strategies. Linear strategies are known to be optimal only for specific models [25, 26, 27, 28, 29, 30]. But in these cases there is no separation of estimation and control.

The results of this paper shed light on the lack of separation in decentralized control of LQG systems. We explain this in Appendix D using the example of decentralized stochastic control with one-step delayed information structure [31, 32, 26]. For this model, we show that the decentralized control problem is equivalent to a MTMSE filtering problem, where the weight matrix depends on the solution of a backward Riccati equation. As shown in Theorem 2, the gains for MTMSE filtering depends on the weight matrix $S$ in the cost function. That is the reason that the computation of the state estimate is not separated from the computation of the controller gains.

V-C Trade-off between filter complexity and estimation accuracy

For graphs with neighborhood sharing information structure, the dimension of $\tilde{I}_{i}^{\mathrm{loc}}(t)$ and $F_{i}(t)$ are proportional to the diameter $\tau^{*}$ of the graph. It is possible to trade-off the implementation complexity with the filtering accuracy by “shedding” information at each agent. We explain this via the example of Sec. IV-B.

We consider two approximate information structures for this example, which we denote by $\{I^{(1)}_{i}(t)\}_{i\in N}$ and $\{I^{(2)}_{i}(t)\}_{i\in N}$. For both these information structures, the common information is the same as before, i.e.,

[TABLE]

But the local information $I^{\mathrm{loc},(m)}_{i}(t)\coloneqq I^{(m)}_{i}(t)\setminus I^{\mathrm{com},(m)}(t)$ is a subset of the original $I^{\mathrm{loc}}_{i}(t)$. In particular, we assume the following.

IS1: In the first approximation, each agent just uses the measurements from a time window of size two to “correct” the common information based estimate, i.e.,

[TABLE] 2. 2.

IS2: In the second approximation, each agent justs uses its local measurements to “correct” the common information based estimate, i.e.,

[TABLE]

For completeness, we refer to the original information structure as IS0. Note that $I^{\mathrm{loc},(m)}_{i}(t)\subset I^{\mathrm{loc}}_{i}(t)$, therefore any filtering strategy based on the approximate information structure $\{I^{(m)}_{i}(t)\}_{i\in N}$ can be implemented in the original information structure $\{I_{i}(t)\}_{i\in N}$. The size of $I^{\mathrm{loc}}_{i}(t)$ (and therefore $\tilde{I}^{\mathrm{loc}}_{i}(t)$) for the different information structures is shown in Table I.

To compare the peformance of these three information structures, we note that the structure of the weight matrix $S$ implies that $\lim_{\lambda\to\infty}J^{*}_{T}/\lambda$ is a constant. So, we evaluate $J^{*}_{T}/\lambda$ for large value of $\lambda$ ($\lambda=100$) and compare the performance of the three information structures. The results are also shown in Table I.

This example shows that it is possible to trade-off the complexity of the MTMSE filter with the estimation accuracy. Note that although the two approximate information structures are almost of the same size, IS1 has better performance than IS2. This is because IS1 uses some local infomration from the neighborhood nodes, while IS2 does not. This suggested that it is better to have some information from many agents rather than a lot of information from a few agents but a more detailed investigation is needed to quantify such a comparison.

VI Conclusion

In this paper, we investigate multi-agent estimation and filtering to minimize team mean-square error. We show that the MTMSE estimates are given by

[TABLE]

The first term of the estimate is the conditional mean of the current state given the common information. The second term may be viewed as a “correction” which depends on the “innovation” in the local measurements. A salient feature of this result is that the gains $\{F_{i}(t)\}_{i\in N}$ depend on the weight matrix $S$. Using illustrative examples, we show that the MTMSE estimates significantly smaller team mean-squared error as compared to MMSE strategy and consensus Kalman filtering.

The results were derived under the assumptions that the state process $\{x(t)\}_{t\geq 1}$ is a linear stochastic process and the observation channels are linear and additive Gaussian noise. In future, we plan to investigate team estimation of general stochastic processes over general measurement channels, which will give rise to non-linear filtering equations.

Finally, our focus in this paper was to establish the structure of MTMSE filtering and filtering strategies. Having identified this structure, it is possible to implement the policy efficiently in a distributed manner. For example, for the infinite horizon setup, it is possible to use a consensus Kalman filter [16, 17, 18, 19, 20, 21] to keep track of the delayed state estimate $\hat{x}(t-\tau^{*}+1)$ and use distributed algorithms to solve the linear system of equations $\bar{\Gamma}\bar{F}=\bar{\eta}$ using distributed algorithms [33, 34, 35].

Appendix A Proof of Theorem 1

A-A A preliminary result

In order to compute the gains and the performance, we need to compute $\hat{\Theta}_{i}=\operatorname{cov}(x,\tilde{y}_{i})$ and $\hat{\Sigma}_{ij}=\operatorname{cov}(\tilde{y}_{i},\tilde{y}_{j})$.

Lemma 5

For any $\{S_{ij}\}_{i,j\in N}$, $\{P_{ij}\}_{i,j\in N}$ and $\{L_{i}\}_{i\in N}$ of compatible dimensions, the following matrix equation

[TABLE]

for unknown $\{F_{i}\}_{i\in N}$ of compatible dimensions can be written in vectorized form as

[TABLE]

where $F$, $\eta$, and $\Gamma$ are as defined in Theorem 1. Furthermore, define $S=[S_{ij}]_{i,j\in N}$ and $P=[P_{ij}]_{i,j\in N}$. If $S>0$, $P\geq 0$, and $P_{ii}>0$, $i\in N$, then $\Gamma>0$ and thus invertible. Then, Eq. (48) has a unique solution that is given by

[TABLE]

□

The proof of Lemma 5 is presented in Appendix B.

A-B Proof of Theorem 1

The key observation behind the proof is that Problem 1 may be viewed as a MTMSE filtering problem [22], where agents observe different information and want to minimize a common estimation cost. For the ease of notation, for a given agent $i$, we let $(g_{i},g_{-i})$ and $(\hat{z}_{i},\hat{z}_{-i})$ denote the strategy and estimates of all agents. Pick an agent $i\in N$, and fix the strategy $g_{-i}$ of all the other agents. Then the expected cost from the point of view of agent $i$ is given by

[TABLE]

where the superscript $g_{-i}$ in the expectation indicates that the cost depends on the strategy of agents other than $i$.

A necessary condition for optimality is that agent $i$ is playing a best response to the strategy of all other players, i.e.,

[TABLE]

It is shown in [22, Theorem 4], that when $c(x,\hat{z})$ is convex, (51) is also a sufficient condition for optimality.

From the dominated convergence theorem, we can interchange the order of derivative and expectation to get

[TABLE]

Substituting the above in (51), we get that a necessary and sufficient condition for a strategy $(g_{i},g_{-i})$ to be team optimal is

[TABLE]

Note here that the superscript $g_{j}$ in $\mathds{E}^{g_{j}}[\hat{z}_{j}\,|\,y_{0},y_{i}]$ highlights that the expectation depends on the choice of $g_{j}$. There is no such dependence in $\mathds{E}[x\,|\,y_{0},y_{i}]$. Thus, the strategy $g$ given by (8) is optimal if and only if

[TABLE]

or equivalently

[TABLE]

Note that from Lemma 1, we have

[TABLE]

Substituting the above and the expression for ${\mathds{E}[\tilde{y}_{j}|y_{0},y_{i}]}$ from Lemma 1 in (54), we get that the strategy given by (8) is optimal if and only if, for all $i\in N$,

[TABLE]

Since the above should hold for all $\tilde{y}_{i}\in\mathds{R}^{d_{y}^{i}}$, the coefficient of $\tilde{y}_{i}$ must be identically zero. Thus, the strategy given by (8) is optimal if and only if

[TABLE]

Furthermore, Lemma 5 implies that when $\hat{\Sigma}_{ii}>0$, then (55) has a unique solution given by (10).

Now for the minimum value of the estimation error, consider a single term of the estimation error

[TABLE]

where $(a)$ follows from substituting (8), $(b)$ uses Lemma 1, and $(c)$ uses the fact that for any matrices $\operatorname{Tr}(ABCD)=\operatorname{Tr}(BCDA)$. Thus, the expected team estimation error is

[TABLE]

where $(d)$ follows from (56), and $(e)$ follows from (55). The result now follows from observing that

[TABLE]

where the first equality follows from \operatorname{Tr}(A^{\mathchoice{\raisebox{0.0pt}{\displaystyle\intercal}}{\raisebox{0.0pt}{\textstyle\intercal}}{\raisebox{0.0pt}{\scriptstyle\intercal}}{\raisebox{0.0pt}{\scriptscriptstyle\intercal}}}B)=\operatorname{vec}(A)^{\mathchoice{\raisebox{0.0pt}{\displaystyle\intercal}}{\raisebox{0.0pt}{\textstyle\intercal}}{\raisebox{0.0pt}{\scriptstyle\intercal}}{\raisebox{0.0pt}{\scriptscriptstyle\intercal}}}\operatorname{vec}(B).

Appendix B Proof of Lemma 5

By vectorizing both sides of (48) and using \operatorname{vec}(ABC)=(C^{\mathchoice{\raisebox{0.0pt}{\displaystyle\intercal}}{\raisebox{0.0pt}{\textstyle\intercal}}{\raisebox{0.0pt}{\scriptstyle\intercal}}{\raisebox{0.0pt}{\scriptscriptstyle\intercal}}}\otimes A)\times\operatorname{vec}(B), we get

[TABLE]

Substituting $\Gamma_{ij}=P_{ij}\otimes S_{ij}$ and $\eta_{i}=\operatorname{vec}(S_{i\bullet}LP_{ii})$, we get (49).

If $S>0$, $P\geq 0$, and $P_{ii}>0$, $i\in N$, then [32, Lemma 1] implies that $\Gamma>0$ and thus invertible. Hence, Eq. (48) has a unique solution that is given by (50).

Appendix C Proof of Theorem 3

$\bar{\Sigma}_{ii}$ is the variance of the innovation in the standard Kalman filtering equation and by positive definiteness of $R_{i}$ is positive definite. Lemma 5 implies that (43) has a unique solution that is given by (44). To show the strategy (41) is optimal, we proceed in two steps. We first identify a lower bound in optimal performance and then show that the proposed strategy achieves that lower bound.

Step 1

From Theorem 2, for any strategy $g$, we have that

[TABLE]

Taking limits of both sides and using Lemma 4 (which implies that $\lim_{t\to\infty}\eta(t)=\bar{\eta}$ and $\lim_{t\to\infty}\Gamma(t)=\bar{\Gamma}$), we get

[TABLE]

Step 2

Suppose $\hat{z}(t)$ is chosen according to strategy (44) and let $J(t)$ denote $\mathds{E}[c(x(t),\hat{z}(t))]$. Following (56) and (57) in the proof of Theorem 1, we have that

[TABLE]

From Lemma 4, we have that

[TABLE]

Thus, by Cesaro’s mean theorem, we get $\lim_{T\rightarrow\infty}\frac{1}{T}\sum_{t=1}^{T}J(t)=J^{*}.$ Hence, the strategy (44) achieves the lower bound of (58) and is therefore optimal.

Appendix D One-step delayed observation sharing

D-A Problem statement

In this section, we use the result of Theorem 2 to show the relationship between MTMSE filtering and control in delayed observation sharing model [31, 32, 26]. The notation used in this section is self-contained and consistent with the standard notation used in decentralized stochastic control.

Consider a decentralized control system with $n$ agents, indexed by the set $N=\{1,\dots,n\}$. The system has a state $x(t)\in\mathds{R}^{d_{x}}$. The initial state $x(1)\sim N(0,\Sigma_{x})$ and the state evolves as follows:

[TABLE]

where $A$ and $B$ are matrices of appropriate dimensions. $u(t)=~{}\operatorname{vec}(u_{1}(t),\cdots,u_{n}(t))$, where $u_{i}(t)\in\mathds{R}^{d_{u}^{i}}$ is the control action chosen by agent $i$, and $\{w(t)\}_{t\geq 1}$, $w(t)\in\mathds{R}^{d_{x}}$ is an i.i.d. process with $w(t)\sim\mathcal{N}(0,\Sigma_{w})$. Each agent observes a noisy version $y_{i}(t)\in\mathds{R}^{d^{i}_{y}}$ of the state given by

[TABLE]

where $\{v_{i}(t)\}_{t\geq 1}$, $v_{i}(t)\in\mathds{R}^{d^{i}_{y}}$, is an i.i.d. process with $v_{i}(t)\sim(0,\Sigma^{i}_{v})$. This may be written in a vector form as

[TABLE]

where $C=\operatorname{rows}(C_{1},\dots,C_{n})$, $v(t)=\operatorname{vec}(v_{1}(t),\dots,v_{n}(t))$, and $y(t)=\operatorname{vec}(y_{1}(t),\dots,y_{n}(t))$.

Assumption 1: The primitive random variables $(x(1),\allowbreak\{w(t)\}_{t\geq 1},\allowbreak\{v_{1}(t)\}_{t\geq 1},\dots,\allowbreak\{v_{n}(t)\}_{t\geq 1})$ are independent.

In addition to its local observation $y_{i}(t)$, each agent also receives the one-step delayed observations of all agents. Thus, the information available to agent $i$ is given by

[TABLE]

Therefore, agent $i$ chooses the control action $u_{i}(t)$ as follows.

[TABLE]

where $g_{i,t}$ is the control laws of agent $i$ at time $t$. The collection $g=(g_{1},\dots,g_{n})$, where $g_{i}=(g_{i,1},\dots,g_{i,T})$ is called the control strategy of the system. The performance of any control strategy $g$ is given by

[TABLE]

where $Q$ is symmetric positive semi-definite matrix, $R$ is symmetric positive definite matrix, and the expectation is with respect to the joint measure on the system variables induced by the choice of $g$.

Problem 4

Given the system dynamics and the noise statistics, choose a control strategy $g$ to minimize the total cost $J(g)$ given by (64).

Problem 4 is a decentralized stochastic control problem. In such problems there is no separation of estimation and control (see, for example [32]). We show that this lack of separation is due to the fact that the MTMSE filtering strategy depends on the weight matrix of the estimation cost.

D-B Equivalence to MTMSE filtering

We start with a basic property of linear quadratic models. Let $P(1{:}T)$ denote the solution to the following backward Riccati equation. $P(T)=Q$ and for $t\in\{T-1,\dots,1\}$,

[TABLE]

Define

[TABLE]

Then, we have the following.

Lemma 6

For any control strategy $g$, define

[TABLE]

Then, a strategy $g$ that minimizes $J^{\circ}(g)$ also minimizes $J(g)$. □

Proof

Following [36, Chapter 8, Lemma 6.1], we can show that the total cost $J(g)$ can be written as

[TABLE]

The third term is equal to $J^{\circ}(g)$ and the first two terms do not depend on the control strategy $g$. Thus, $J(g)$ and $J^{\circ}(g)$ have the same argmin. ■

Now, we split the state $x(t)$ into a deterministic part $\bar{x}(t)$ and a stochastic part $\tilde{x}(t)$ as follows. $\bar{x}(1)=0,\quad\tilde{x}(1)=x(1),$ and

[TABLE]

Since the system is linear, we have

[TABLE]

Note that $\bar{x}(t)$ is a function of the past control actions, which are known to all agents. Now, for any control strategy $g$, define $\hat{z}_{i}(t)=u_{i}(t)+L_{i}(t)\bar{x}(t)$. Then, the cost $J^{\circ}(g)$ may be written as

[TABLE]

The process $\{\tilde{x}(t)\}_{t\geq 1}$ is an uncontrolled linear stochastic process and the cost (67) is of of the same form as the weighted mean-square cost that we have considered in this paper.

Following [25], we define $\tilde{I}_{i}(t)=\{\tilde{y}_{i}(t),\tilde{y}(1{:}t-1)\}$ which may be considered as the control-free part of the information structure.

Lemma 7

For any strategy $g$ and any agent $i\in N$, $\tilde{I}_{i}(t)$ is equivalent to $I_{i}(t)$, i.e., they generate the same sigma algebra. □

Proof

The result follows from a similar argument as given in [37, Chapter 7, Section 3]. ■

Since $\tilde{I}_{i}(t)$ is equivalent to $I_{i}(t)$, we may assume that $\hat{z}_{i}(t)$ is chosen as a function of $\tilde{I}_{i}(t)$ instead of $I_{i}(t)$. Thus, Problem 4 is equivalent to the following MTMSE filtering problem.

Problem 5

Suppose $n$ agents observe the linear dynamical system $\{\tilde{x}(t)\}_{t\geq 1}$ and share their observations over a one-step delayed sharing communication graph. Thus, the information available at agent $i$ is

[TABLE]

Agent $i$ chooses an estimate $\hat{z}_{i}(t)$ of $\tilde{x}(t)$ according to an estimation strategy $h_{i,t}$, i.e.,

[TABLE]

to minimize an estimation cost given by (67).

Problem 5 is a MTMSE filtering problem and can be solved using Theorem 2. One can then take the solution of Problem 5 and translate it back to Problem 4 as follows.

Theorem 4

Let $h^{*}$ be the optimal strategy for Problem 5, i.e.,

[TABLE]

where

[TABLE]

and the gains $\{F_{i}(t)\}$ are computed as per Theorem 2. Define strategy $g^{*}$ as follows:

[TABLE]

i.e.,

[TABLE]

where $\hat{x}(t)=\mathds{E}[x(t)|I^{\mathrm{com}}(t)]=\bar{x}(t)+\mathds{E}[\tilde{x}(t)|\tilde{y}(1{:}t-1)]$. Then $g^{*}$ is the optimal strategy for Problem 4. □

Proof

The change of variables $\hat{z}_{i}(t)=u_{i}(t)+L_{i}(t)\bar{x}(t)$ implies that if $h^{*}$ is an optimal strategy for Problem 5, then $g^{*}$ given by (69) is optimal for Problem 4.

To establish (70), we need to show that $\hat{x}(t)=\bar{x}(t)+\hat{\tilde{x}}(t)$. Define, $I^{\mathrm{com}}(t)=\{y(1{:}t-1),u(1{:}t-1)\}$ and $\tilde{I}^{\mathrm{com}}(t)=\{\tilde{y}(1{:}t-1)\}$. Then by Lemma 7 we have, $I^{\mathrm{com}}(t)$ is equivalent to $\tilde{I}^{\mathrm{com}}(t)$, i.e., they generate the same sigma algebra. The rest of the proof follows from the definition of $\hat{x}(t)$. We have

[TABLE]

where $(a)$ follows from state splitting and $I^{\mathrm{com}}(t)=\tilde{I}^{\mathrm{com}}(t)$ and $(b)$ follows from the fact that $\bar{x}(t)$ is a deterministic function of $I^{\mathrm{com}}(t)$. ■

The main take away is as follows. By a simple change of variables we showed that the one-step delayed observation sharing problem is equivalent to a MTMSE filtering problem, where the weight matrix $S(t)$ of the estimation cost depends on the backward Riccati equation for the cost function. The MTMSE filtering strategy depends on the weight matrix $S(t)$ and that is the reason why there is no separation between estimation and control. Nonetheless, the optimal gains can be computed as follows.

Solve a Riccati equation to compute the weight functions $S(1{:}T)$ and gains $L(1{:}T)$. 2. 2.

Solve a Kalman filtering equation (which does not depend on $S(1{:}T)$) to compute the covariances $\hat{\Sigma}(t)$ and $\hat{\Theta}(t)$ defined in Theorem 2. 3. 3.

Use $S(t)$, $L(t)$, $\hat{\Sigma}(t)$, and $\hat{\Theta}(t)$ to obtain the optimal gains $F_{i}(t)$ by solving a system of matrix equations. 4. 4.

Using Theorem 4 above, we can write the optimal strategy $g^{*}_{i,t}$ in terms of $F_{i}(t)$ and $L_{i}(t)$.

Acknowledgment

The authors are grateful to Peter Caines, Roland Malhame, and Demosthenis Teneketzis for useful discussion and feedback.