Second Order Statistics Analysis and Comparison between Arithmetic and Geometric Average Fusion

TL;DR

This paper compares the second order statistics of arithmetic and geometric averages in data fusion for target tracking, analyzing their variance and mean square error in different fusion contexts, including complex Gaussian mixtures.

Contribution

It provides a detailed analysis and comparison of second order statistics for AA and GA in v-fusion and f-fusion, including complex Gaussian mixture scenarios.

Findings

GA generally has lower variance than AA in v-fusion.

Differences between AA and GA are more pronounced in f-fusion with Gaussian mixtures.

The analysis includes exact derivations and illustrative examples.

Abstract

Two fundamental approaches to information averaging are based on linear and logarithmic combination, yielding the arithmetic average (AA) and geometric average (GA) of the fusing initials, respectively. In the context of target tracking, the two most common formats of data to be fused are random variables and probability density functions, namely $v$-fusion and $f$-fusion, respectively. In this work, we analyze and compare the second order statistics (including variance and mean square error) of AA and GA in terms of both $v$-fusion and $f$-fusion. The case of weighted Gaussian mixtures representing multitarget densities in the presence of false alarms and misdetection (whose weight sums are not necessarily unit) is also considered, the result of which appears significantly different from that for a single target. In addition to exact derivation, exemplifying analysis and illustrations are provided.

Equations

$${\bar{\theta }}_i\triangleq {\mathrm{E}}_{f_{{\hat{\theta }}_i}}[{\hat{\theta }}_i]={\int }_{\Theta }x{f}_{{\hat{\theta }}_i}(x)dx=\theta .$$

$$\Sigma_{\hat{\mathbf{\theta}}_{i}}\triangleq\int_{\mathcal{\Theta}}\big{(}x-\bar{\mathbf{\theta}}_{i}\big{)}^{2}f_{\hat{\mathbf{\theta}}_{i}}(x)dx=\int_{\mathcal{\Theta}}x^{2}f_{\hat{\mathbf{\theta}}_{i}}(x)dx-\big{(}\bar{\mathbf{\theta}}_{i}\big{)}^{2}\hskip 1.42262pt.$$

$$\mathrm{mse}({\hat{\theta }}_i)\triangleq {\mathrm{E}}_{f_{{\hat{\theta }}_i}(x)}[(\theta -{\hat{\theta }}_i{)}^2]={\int }_{\Theta }(\theta -x{)}^2{f}_{{\hat{\theta }}_i}(x)dx.$$

$$\mathrm{mse}({\hat{\theta }}_i)={\Sigma }_{{\hat{\theta }}_i}+({\bar{\theta }}_i-\theta {)}^2.$$

$$\mathrm{supp}(f)=\{x\in X|f(x)\ne 0\}.$$

$${\hat{\theta }}_v^{\mathrm{AA}}\triangleq \sum _{i\in \mathcal{I}}{\omega }_i{\hat{\theta }}_i.$$

$${\hat{\theta }}_v^{\mathrm{GA}}\triangleq \prod _{i\in \mathcal{I}}{{\hat{\theta }}_i}^{{\omega }_i}.$$

$$\log {\hat{\theta }}_v^{\mathrm{GA}}=\sum _{i\in \mathcal{I}}{\omega }_i\log {\hat{\theta }}_i.$$

$$f_{{\hat{\theta }}_{\mathrm{AA}}}(x)=\sum _{i\in \mathcal{I}}{\omega }_i{f}_{{\hat{\theta }}_i}(x),$$

$$f_{\hat{\theta}_{\mathrm{GA}}}(x)=C^{-1}\prod_{i\in\mathcal{I}}\big{(}f_{\hat{\theta}_{i}}(x)\big{)}^{\omega_{i}}\hskip 1.42262pt,$$

$${\bar{\theta }}_i=\theta ,\forall i\in \mathcal{I}.$$

$${\bar{\theta }}_v^{\mathrm{AA}}=\sum _{i\in \mathcal{I}}{\omega }_i{\bar{\theta }}_i=\theta .$$

$${\Sigma }_{{\hat{\theta }}_v^{\mathrm{AA}}}$$

$$=\sum _{i\in \mathcal{I}}{\omega }_i^2{\Sigma }_{{\hat{\theta }}_i}+\sum _{i<j\in \mathcal{I}}2{\omega }_i{\omega }_j\mathrm{Cov}({\hat{\theta }}_i,{\hat{\theta }}_j).$$

$$\rho \triangleq \frac{\mathrm{Cov}({\hat{\theta }}_1,{\hat{\theta }}_2)}{\sqrt{{\Sigma }_{{\hat{\theta }}_1}{\Sigma }_{{\hat{\theta }}_2}}},$$

$${\Sigma }_{{\hat{\theta }}_v^{\mathrm{AA}}}={\omega }_1^2{\Sigma }_{{\hat{\theta }}_1}+{\omega }_2^2{\Sigma }_{{\hat{\theta }}_2}+2{\omega }_1{\omega }_2\rho \sqrt{{\Sigma }_{{\hat{\theta }}_1}{\Sigma }_{{\hat{\theta }}_2}}.$$

$${\Sigma }_{{\hat{\theta }}_v^{\mathrm{AA}}}\le \max ({\Sigma }_1,{\Sigma }_2),$$

$$\alpha \triangleq \frac{{\Sigma }_2}{{\Sigma }_1}.$$

$$h(w;\alpha ,\rho )\triangleq 1-2w+w^2+w^2\alpha +2\rho {\alpha }^{\frac{1}{2}}(w-w^2).$$

$${\omega }_1=\frac{\alpha -\rho {\alpha }^{\frac{1}{2}}}{1+\alpha -2\rho {\alpha }^{\frac{1}{2}}},{\omega }_2=\frac{1-\rho {\alpha }^{\frac{1}{2}}}{1+\alpha -2\rho {\alpha }^{\frac{1}{2}}}.$$

$${\Sigma }_{{\hat{\theta }}_v^{\mathrm{AA}}}\ge \frac{\alpha (1-{\rho }^2)}{1+\alpha -2\rho {\alpha }^{\frac{1}{2}}}{\Sigma }_1.$$

$${\Sigma }_{{\hat{\theta }}_v^{\mathrm{AA}}}\ge \min ({\Sigma }_1,{\Sigma }_2),$$

$$\frac{\mathrm{Cov}(\hat{\theta}_{1},\hat{\theta}_{2})}{\sqrt{\Sigma_{\hat{\theta}_{1}}\Sigma_{\hat{\theta}_{2}}}}<\bigg{(}\frac{\min\big{(}\Sigma_{1},\Sigma_{2}\big{)}}{\max\big{(}\Sigma_{1},\Sigma_{2}\big{)}}\bigg{)}^{\frac{1}{2}}\hskip 1.42262pt,$$

$${\Sigma }_{\log {\hat{\theta }}_v^{\mathrm{GA}}}$$

$$=\sum _{i\in \mathcal{I}}\sum _{j\in \mathcal{I}}{\omega }_i{\omega }_j\mathrm{Cov}(\log {\hat{\theta }}_i,\log {\hat{\theta }}_j).$$

$$\displaystyle\mathrm{mse}\big{(}\hat{\theta}_{v}^{\mathrm{AA}}\big{)}=$$

$$=$$

$$\gamma \triangleq \frac{\mathrm{mse}({\hat{\theta }}_2)}{\mathrm{mse}({\hat{\theta }}_1)}.$$

$$\beta<\bigg{(}\frac{\min\big{(}\mathrm{mse}(\hat{\theta}_{1}),\mathrm{mse}(\hat{\theta}_{2})\big{)}}{\max\big{(}\mathrm{mse}(\hat{\theta}_{1}),\mathrm{mse}(\hat{\theta}_{2})\big{)}}\bigg{)}^{\frac{1}{2}}\hskip 1.42262pt,$$

$$h\Big{(}\frac{1}{2}\Big{)}=\frac{1+\gamma+2\beta\gamma^{\frac{1}{2}}}{4}\hskip 1.42262pt.$$

$$\beta <\frac{3-\gamma }{2{\gamma }^{\frac{1}{2}}}\triangleq g(\gamma ).$$

$$\frac{\mathrm{d}g(\gamma )}{\mathrm{d}\gamma }=\frac{-1-3{\gamma }^{-1}}{4{\gamma }^{\frac{1}{2}}}<0,$$

Full text

††journal: Elsevier

1 Introduction

The rapid development and extensive deployment of sensor/agent networks, have stemmed remarkable interest in distributed data fusion which demonstrates evident advantages in many aspects. For example, in the context of target tracking using a decentralized sensor network, the sensor cooperation can compensate for the effect of the mis-detection, false-alarms and even the failure of the local sensor and extends their fields of view, eventually resulting in improved estimation accuracy and improved robustness [1, 2, 3, 4, 5, 6]. Particular interest in distributed data fusion has been paid to calculating the “average” over the information owned by locally netted sensors/agents via peer-to-peer communication in an efficient, flexible and scalable way [7, 8, 9, 6, 10]. Fundamentally, the average can be defined in two manners including, the arithmetic average (AA) and the geometric average (GA). Simply put, the former is a type of linear/convex fusion, akin to the linear opinion pool approach, while the latter is nonlinear/logarithmic fusion akin to the logarithmic opinion pool approach [11, 12], or to say, linear versus log-linear pools [13].

In the context of multi-sensor/multi-agent target tracking, the two most important types of information for fusion among local sensors/agents are random variables (representing parameters such as the number of targets, clutter rate, target existing probability, etc.) and probability density functions (PDFs), for which the fusion is referred to as v-fusion and f-fusion, respectively. While it seems that the AA fusion is more common in the former [7, 8, 4, 9], the GA fusion is vibrant in the latter [3, 14, 15], which coincides with the Chernoff fusion [16, 17, 18, 19] and is also known as covariance intersection (CI) when Gaussian functions that are uniquely characterized by the first and second order statistics are particularly considered [20, 21, 22, 23, 24, 25]. The CI approach was originally proposed for fusing unknown-correlated estimates produced at distinct but not necessarily independent sensors to avoid information double accounting in the fusion. Likewise, the AA fusion can also avoid information double accounting [22]. Further approaches to combining probability distributions of unknown cross-correlation can be found in the literature [26, 27, 28, 29, 30]. In particular, a so-called generalized mean fusion approach is proposed in [30] which includes both AA and GA in a unified expression. It is interesting to note (for discrete f-fusion) that “a linear opinion pool is the distribution that minimizes the sum of KL [Kullback-Leibler] measures from the expert distributions to the aggregate distribution. A log-linear opinion pool is one that minimizes the sum of KL measures from the aggregate distribution to the expert distributions. [13]” This nice property for the GA fusion was earlier pointed out in [11] and later extended to the PHD in [31].

In addition to the posterior PDF, the fusing functions can also be the likelihood functions [32, 15, 33] or the probability hypothesis density (PHD) functions [14, 34, 31, 35, 36, 37]. (The PHD [38] differs from the PDF in that its integral over any region gives the expected number of targets in that region which can be any real number.) In comparison, the AA fusion has also been applied for PHD fusion [39, 40, 41, 42, 43, 44, 45, 46] and for raw data fusion in the means of clustering [47, 48]. Both averaging approaches to data fusion have demonstrated, either theoretically or experimentally, gains in estimation accuracy and/or robustness, whereas deficiencies have also been identified from different viewpoints [11, 12, 34, 49, 50, 41, 43, 44, 51, 52, 36, 30].

Despite a few analysis about the variance alone [21, 22, 14, 49] in f-fusion, the mean square error (MSE) of the AA [45] in v-fusion and the divergence to the ideal fusion rule in f-fusion [30], comprehensive analysis and comparison of their statistics (including the variance and MSE) are still lacking. What is missing and interesting also includes the specific consideration of the multitarget density fusion in the presence of false alarms and misdetection.

In this paper, we are not intended to investigate the motivation behind both approaches, or propose any new algorithms. Rather, we analyze and compare the statistics of the GA and of the AA in the viewpoint of point estimation, with respect to v-fusion and f-fusion, respectively. In addition to a detailed formal analysis, some approximations and exemplifying illustrations are also given. Further, we restrict our discussion in the scalar real space $\mathbb{R}$ for simplicity, although most of the conclusions can be extended to the multidimensional case.

The paper is organized as follows. Preliminaries are briefly introduced in Section 2. Our major analysis of the v-fusion and of the f-fusion is given in sections 3 and 4, respectively. A hybrid use of both approaches for averaging PHDs is discussed in Section 5, as well as further comparison between v-fusion and f-fusion. Key findings are summarized by Remarks throughout the main body of the paper and in Section 6.

2 Preliminaries

2.1 Definitions

For the unknown parameter $\theta$ of interest that takes value in a measurable region $\mathcal{\Theta}\!\subseteq\!\mathbb{R}$, the estimator $\hat{\theta}_{i}$ associated with PDF $f_{\hat{\mathbf{\theta}}_{i}}(x)$, is unbiased if it yields, on average, the true value of the parameter [53, Section 2.3], i.e.,

[TABLE]

The variance of the estimator $\hat{\mathbf{\theta}}_{i}$ is given by

[TABLE]

The MSE of an estimator $\hat{\mathbf{\theta}}_{i}$ is given by [53, Section 2.4]

[TABLE]

A straightforward expansion of (3) will lead to

[TABLE]

That is, the MSE of an estimator equals the sum of its variance and the square of its bias (if any).

Furthermore, suppose that $f:X\rightarrow\mathbb{R}$ is a real-valued function whose domain is a set $X$. The set-theoretic support of $f$, denoted as $\mathrm{supp}(f)$, is the set of points in $X$ where $f$ is non-zero, i.e.,

[TABLE]

2.2 Averaging in Terms of Variables: v-fusion

Let us consider estimators $\hat{\theta}_{i},i\in\mathcal{I}\subseteq\mathbb{N}$ given in terms of random variables, such as the estimate of the number of targets [45] or the target existing probability. There is no PDF or uncertainty information available and so only point estimates that are random variables are involved. The variable-AA is given by

[TABLE]

Hereafter, the fusing weights are limited as $\omega_{i}\in(0,1)$, $\sum_{i\in\mathcal{I}}{\omega_{i}}=1$. (Obviously, $\omega_{i}=0$ indicates that information $i$ does not really get involved in the fusion.)

In contrast to (6), the variable-GA is given by

[TABLE]

Note that, the variable-GA fusion may lead to an imaginary number when the fusing variable is negative, which is beyond the consideration of this work.

Obviously, the GA fusion amounts to the AA fusion on the logarithms of the variables, namely (cf. (6))

[TABLE]

2.3 Averaging in Terms of PDFs: f-fusion

When the local estimator is given as a function such as a PDF or a PHD, the f-fusion is involved. Given estimators $\hat{\theta}_{i}$ with PDFs $f_{\hat{\theta}_{i}}(x)$, $i\in\mathcal{I}$, the PDF-AA is given by

[TABLE]

and the PDF-GA is given by

[TABLE]

where C\triangleq\int_{\mathcal{\Theta}}\prod_{i\in\mathcal{I}}\big{(}f_{\hat{\theta}_{i}}(x)\big{)}^{\omega_{i}}dx is a normalization term to ensure the result being a PDF (if possible). (In the generalized GA fusion applied to the PHDs [14, 34], such a normalization may be unnecessary. But then, the fused result of two PDFs may not be a PDF [34]. )

Fundamentally, the support of the AA fusion PDF is the union of those of all initial PDFs. In contrast, the support of $f_{\hat{\theta}_{\mathrm{GA}}}(x)$ is the intersection of those of all initial PDFs, which may be empty. We leave here further discussion on this point and we assume all fusing estimators have the same support unless otherwise stated.

3 Statistics Analysis for v-Fusion

3.1 Bias Analysis

Consider unbiased estimators $\hat{\theta}_{i},i\in\mathcal{I}$ given as random variables. The unbiasedness indicates (cf. (1))

[TABLE]

Combining (6) and (11) yields

[TABLE]

That is, the AA retains the unbiasedness (namely it remains unbiased if all fusing initials are unbiased.)

However, as addressed the variable GA fusion may lead to an imaginary result if any $\hat{\theta}_{i}$ is an imaginary number and so we cannot compare it with the AA mean in general. Even in the case that $\hat{\theta}_{i},\forall i\in\mathcal{I}$ are all positive, the inequality of arithmetic and geometric means [54, Chapter 2] indicates $\bar{\theta}_{v}^{\mathrm{GA}}\leq\bar{\theta}_{v}^{\mathrm{AA}}$  . Here, the two means are equal if and only if ${\theta}_{i}={\theta}_{j},\forall i,j\in\mathcal{I}$. In a nutshell, the AA but not the GA retains unbiasedness in general.

3.2 Variance Analysis

The variance of a weighted sum of multiple variables is given by the weighted sum of their covariances, [55], i.e.,

[TABLE]

Here, $\mathrm{Cov}(\hat{\theta}_{1},\hat{\theta}_{2})$ denotes the covariance between $\hat{\theta}_{1}$ and $\hat{\theta}_{2}$.

Let us consider two variables for simplicity and define the correlation coefficient [56, Chapt. 4] as

[TABLE]

taking values between $-1$ and $1$. (For two independent variables, $\rho=0$ and for two identical variables, $\rho=1$.) Then, (3.2) reduces to

[TABLE]

We now analyze the bounds of $\Sigma_{\hat{\theta}_{v}^{\mathrm{AA}}}$. First, it is easy to verify that \Sigma_{\hat{\theta}_{v}^{\mathrm{AA}}}\leq\big{(}\omega_{1}\sqrt{\Sigma_{1}}+\omega_{2}\sqrt{\Sigma_{2}}\big{)}^{2}\leq\mathrm{max}(\Sigma_{1},\Sigma_{2}), indicating that

[TABLE]

where the upper bound is approached when $w_{1}\to 0$ (if $\Sigma_{1}\leq\Sigma_{2}$) or $w_{1}\to 1$ (if $\Sigma_{1}\geq\Sigma_{2}$). To derive the lower bound of $\Sigma_{\hat{\theta}_{v}^{\mathrm{AA}}}$, we further define

[TABLE]

Due to the symmetry of the expression of $\Sigma_{\hat{\theta}_{v}^{\mathrm{AA}}}$, we only need to consider $\alpha\geq 1$ in our analysis; the results hold by exchanging $\Sigma_{2}$ with $\Sigma_{1}$ if $\alpha<1$. Our analysis is based on a convex function of $w\in(0,1)$ as follows

[TABLE]

Bound analysis of the function $h(w;\alpha,\rho)$ is given in Appendix A. Hereafter, we write $h(w;\alpha,\rho)$ as $h(w)$ for short. So, we have $\Sigma_{\hat{\theta}_{v}^{\mathrm{AA}}}=h(\omega_{2})\Sigma_{1}$. If $\rho<\alpha^{-\frac{1}{2}}$, the fusing weights that minimize $h(w)$ are given by (cf. (55))

[TABLE]

This corresponds to the lower bound of $\Sigma_{\hat{\theta}_{v}^{\mathrm{AA}}}$ as 111In the case of $\rho=0$, the two fusing weights as in (19) are given by the inverse of the opposite variance divided by the sum of the two inverses. The optimal fusion reduces to the classical Millman’s equation [57], analogous to the weighted least squares fusion of two independent estimators.

[TABLE]

Otherwise (if $\rho\geq\alpha^{-\frac{1}{2}}$), the lower bound is given by

[TABLE]

where the bound is approached when $w_{1}\to 1$ (if $\Sigma_{1}\leq\Sigma_{2}$) or $w_{1}\to 0$ (if $\Sigma_{1}\geq\Sigma_{2}$).

Remark 1. For v-fusion, the variance of the AA is upper bounded by the greatest variance of the fusing estimators. Its lower bound as given in (20) is smaller than the smallest variance of the fusing estimators if the correlation coefficient between two fusing estimator satisfies

[TABLE]

otherwise, the lower bound is given by the smallest variance of the fusing estimators.

Notably, when these two variables are inversely correlated (namely $\mathrm{Cov}(\hat{\theta}_{1},\hat{\theta}_{2})<0$), (22) always holds. On the other hand, to calculate $\Sigma_{\hat{\theta}_{v}^{\mathrm{GA}}}$, substituting $\hat{\theta}_{v}^{\mathrm{AA}},\hat{\theta}_{i},i\in\mathcal{I}$ in (3.2) with $\mathrm{log}\hat{\theta}_{v}^{\mathrm{GA}},\mathrm{log}\hat{\theta}_{i},i\in\mathcal{I}$ (cf. (8)), respectively, yields

[TABLE]

The above formulation involves the calculation of the covariance between (logarithmic) functions of two random variables, which can be addressed in terms of the cumulative distribution function; see, e.g., [58]. We omit further analysis on this mathematical problem, but instead, to gain insight and to illustratively compare between the AA and the GA, we study two representative examples by means of the Monte Carlo simulation.

3.2.1 Numerical analysis for Gaussian v-fusion

Note that the GA of two Gaussian variables is no longer a Gaussian variable (unless two fusing variables are identical). For numerical illustration, we here consider two approximate Gaussian distributions with $\mu_{1}=50,\Sigma_{1}=100$, and with $\mu_{2}=60,\Sigma_{2}=200$, respectively, in which the negative support of the Gaussian PDF (which is actually ignorable in the given examples as the negative part is far more than four-sigma to the mean of the distribution) is truncated and so all samples are guaranteed to be positively valued in order to avoid the imaginary number problem of the GA fusion.

Two groups of samples are generated with different correlation coefficient $\rho$s. The corresponding values of the mean and variance of the AA and of the GA are given in Fig. 1. It is confirmed that, when $\rho=0.70846>\alpha^{-\frac{1}{2}}$ (as $\alpha=\frac{\Sigma_{2}}{\Sigma_{1}}=2$), we obtain dual bounds of $\Sigma_{\hat{\theta}_{v}^{\mathrm{AA}}}$ as in (21) otherwise $\Sigma_{\hat{\theta}_{v}^{\mathrm{AA}}}$ can be smaller than the smallest variance of the fusing estimators. It is further seen that,

Remark 2. The variance of the AA can be either greater or smaller than that of the GA when different fusing weights are used. There is a cross-over of their values (namely the smaller becomes the greater) as $\omega_{1}$ increases from 0 to 1. For certain $\rho$ and $\alpha$, the lowest AA variance that can be yielded by adjusting the fusing weights is never greater than that of the GA .

3.2.2 Numerical analysis for Poisson v-fusion

We further consider two Poisson variables $\hat{\theta}_{1}\sim\mathrm{Poisson}(\lambda_{1})$ and $\hat{\theta}_{2}\sim\mathrm{Poisson}(\lambda_{2})$, where $\lambda_{1}=12$ and $\lambda_{2}=10$ are the Poisson rates which indicate both the mean and variance of the variable. The Poisson variable is important in the tracking community, e.g., the number of targets or of false alarms that appear at a given time-interval is often modeled as a Poisson variable [38, 59]. Note that both AA and GA of two Poisson variables are no longer Poisson.

Once more, we use the Monte Carlo method for numerical approximation. The means and variances of the AA and GA of two Poisson random variables under different correlation coefficient $\rho$s and fusing weights are given in Fig. 2. The results are highly consistent to what shown in the Gaussian case (cf. Fig. 1) and so the statement given in Remark 2 still holds.

3.3 MSE Analysis for AA and Numerical Comparison to GA

In this section, we study the MSE of the AA and compare it with that of the GA numerically, based on general variables that may be correlated.

Inserting (6) in (3) yields

[TABLE]

where \beta\triangleq\frac{\mathrm{E}_{f_{\hat{\theta}_{\mathrm{AA}}}(x)}\big{[}(\theta-\hat{\theta}_{1})(\theta-\hat{\theta}_{2})\big{]}}{\sqrt{\mathrm{mse}(\hat{\theta}_{1})\mathrm{mse}(\hat{\theta}_{2})}}\in(-1,1).

As addressed, the fractional order of a Gaussian variable may involve imaginary numbers. Therefore, we cannot simply get the MSE of the GA for v-fusion. To overcome this, once more, by means of the Monte Carlo simulation, we consider two approximate Gaussian variables $\hat{\theta}_{1}(x)\sim\mathcal{N}(x;50,100)$ and $\hat{\theta}_{2}(x)\sim\mathcal{N}(x;60,200)$, for which we simulate three different real variables $\theta=45,55$, and $65$, respectively, for different $\beta$s. The v-fusion results are shown in Fig. 3 for the case of two independent variables and in Fig. 4 for the case of two correlated variable with correlation coefficient $\rho=0.70736$. It is seen that

Remark 3. *The MSE of the AA can be either greater or smaller than that of the GA when different fusing weights are used. The greatest discrepancy between them occurs when the fusing weights are somewhere in the scope (0,1). The lowest bound of the MSE of either the AA or the GA is their corresponding variances, which are obtained when the AA/GA turns out to be unbiased which is only possible when the real parameter lies between two variables. Accordingly, the lower bound of the MSE of the AA is smaller than that of the GA. *

3.3.1 Bounds and Comparison

To gain analytic results on the MSE of the AA for v-fusion, let us define

[TABLE]

Then, it can be easily varified that \mathrm{mse}\big{(}\hat{\theta}_{v}^{\mathrm{AA}}\big{)}=h(\omega_{2})\mathrm{mse}(\hat{\theta}_{1}), where $h(w)$ is defined in (18) (with $\rho$ and $\alpha$ replaced by $\beta$ and $\gamma$, respectively). Therefore, analogous to our analysis in Section 3.2, lower and upper bounds of \mathrm{mse}\big{(}\hat{\theta}_{v}^{\mathrm{AA}}\big{)} can be obtained by using the same fusing weights $\omega_{1}$ and $\omega_{2}$ as in (55). Akin to Remark 1, we obtain:

Remark 4. For v-fusion, the upper bound of the MSE of the AA is given by the greatest MSE of the fusing estimators. The lower bound is smaller than the smallest MSE of the fusing estimators if the correlation between two fusing estimators satisfies

[TABLE]

*otherwise, the lower bound is given by the smallest MSE of the fusing estimators. *

Notably, when the real parameter $\theta$ lies on or between $\bar{\theta}_{1}$ and $\bar{\theta}_{2}$ (namely \mathrm{E}_{\mathcal{\Theta}}\big{[}(\theta-\hat{\theta}_{1})(\theta-\hat{\theta}_{2})\big{]}\leq 0 and so $\beta\leq 0$), (26) always holds.

3.3.2 Unweighted AA

The MSE is a key metric in evaluating an estimator/tracker [53]. However, in practice, $\gamma$ is often unknown since the MSE of each fusing estimator that is calculated based on the real parameter is practically unknown. 222 In the literature, e.g., [21, 23, 35, 24, 25, 43], the most common approach to designing the fusing weights is based on minimizing the (trace or determinant of) variance of the fused estimator, which only equals the MSE when the estimator is unbiased. However, the GA does not guarantee unbiasedness as addressed. Such a minimum variance criterion for selecting the weights can be dated at latest back to [60]. One may simply choose to use uniform fusing weights $\omega_{1}=\omega_{2}=\frac{1}{2}$ (namely unweighted averaging). Then, we obtain (cf. (18))

[TABLE]

In this case, a sufficient and necessary condition for the un-weighted AA fusion to be “better” than the best fusing estimator in the sense of obtaining smaller MSE (i.e., h\big{(}\frac{1}{2}\big{)}<1) is given by (if possible)

[TABLE]

Calculating the derivative of $g(\gamma)$ with respect to $\gamma$ yields

[TABLE]

which indicates that $g(\gamma)$ decreases with the increase of $\gamma$, and therefore, $g(\gamma)<-1,\forall\gamma>9$. Since $-1<\beta$, we therefore assert that h\big{(}\frac{1}{2}\big{)}<1 is impossible if $\gamma>9$. In this case, the MSE of the unweighted AA must lie between the best and the worst of the MSEs of the fusing estimators.

4 Statistics Analysis for f-Fusion

In the context of Bayesian estimation, the local estimate is given in the form of a posterior PDF or PHD which contains the complete information about the distribution of the state of the interest. Then, the f-fusion as given in (9) and (10) is needed, which as to be shown in this section is very different from the v-fusion.

4.1 Bias Analysis

Let us consider again a set of unbiased fusing estimators with PDFs as given in (11), we obtain from (9)

[TABLE]

That is, the AA retains the unbiasedness in the f-fusion. In contrast,

[TABLE]

which does not equal $\theta$ in general.

In particular, (31) does not equal $\theta$ if any $f_{\hat{\theta}_{i}}(x),{i\in\mathcal{I}}$ is asymmetric (which is common when the function is represented by a mixture/sum of weighted sub-functions, such as the widely used Gaussian mixture (GM)). We demonstrate this by a simple example in Appendix B. (We must note that, the above bias analysis rooted in the classic point estimation is different from the interpretation of the unbiasedness for an estimator in the Bayesian view 333In the Bayesian formulation, the real parameter $\theta$ is considered random and the Bayesian posterior is given in the manner of an estimate to the true distribution of $\theta$; so the unbiasedness definition as in (1) does not directly apply to the Bayesian estimator in general; see, e.g., [61]. In the Bayesian view, one care about the quality of the distribution rather than a single point.)

We add that the estimate bias can greatly reduce the probability for the fusion/filter to benefit [62] in time series estimation since the bias is supposed to progagrate over time.

4.2 Variance Analysis (for Two Gaussian PDFs)

In this section, we analyze the variances of the PDF-AA $f_{\hat{\theta}_{\mathrm{AA}}}(x)$ and PDF-GA $f_{\hat{\theta}_{\mathrm{GA}}}(x)$ for fusing two Gaussian PDFs $f_{\hat{\theta}_{1}}(x)=\mathcal{N}(x;\mu_{1},\Sigma_{1})$ and $f_{\hat{\theta}_{2}}(x)=\mathcal{N}(x;\mu_{2},\Sigma_{2})$.

4.2.1 General Result

In the addressed case, (9) reduces to a GM-PDF $f_{\hat{\theta}_{f}^{\mathrm{AA}}}(x)=\omega_{1}\mathcal{N}(x;\mu_{1},\Sigma_{1})+\omega_{2}\mathcal{N}(x;\mu_{2},\Sigma_{2})$ whose mean $\bar{\theta}_{f}^{\mathrm{AA}}$ and variance $\Sigma_{\hat{\theta}_{f}^{\mathrm{AA}}}$ are

[TABLE]

respectively, where $\Delta(\omega_{1},\omega_{2})\triangleq\omega_{1}\omega_{2}(\mu_{1}-\mu_{2})^{2}\geq 0$.

In contrast, the GA of two Gaussian PDFs remains a Gaussian PDF. That is, (10) reduces to a single Gaussian PDF $f_{\bar{\theta}_{f}^{\mathrm{GA}}}(x)=\mathcal{N}(x;\bar{\theta}_{f}^{\mathrm{GA}},\Sigma_{\hat{\theta}_{f}^{\mathrm{GA}}})$ with [14]

[TABLE]

As shown, both the mean of the AA as in (32) and the mean of the GA as in (35) show a linear combination of the means of the fusing estimators. (In this sense, the CI approach is also considered as a linear fusion of estimators [23, 24, 25, 29].) Differently, the variances of the fusing estimators are also involved in the latter but not in the former. In what follows, we analyze and compare their variances as in (33) and (34).

4.2.2 Bounds and Comparison

Given $0<\omega_{1},\omega_{2}<1$, we obtain the dual, tight bounds on $\Sigma_{\hat{\theta}_{f}^{\mathrm{GA}}}$ from (34)

[TABLE]

where the equations hold when and only when $\Sigma_{1}=\Sigma_{2}$ for which $\Sigma_{\hat{\theta}_{f}^{\mathrm{GA}}}=\Sigma_{1}=\Sigma_{2}$, regardless of the fusing weights. Otherwise, if $\Sigma_{1}\neq\Sigma_{2}$, the bounds are approached when $\omega_{1}\to 0,\omega_{2}\to 1$ (for one of the dual bounds) or when $\omega_{1}\to 1,\omega_{2}\to 0$ (for the other bound).

Since $\Delta(\omega_{1},\omega_{2})\geq 0$, we obtain

[TABLE]

where, $\mathrm{LB}(\Sigma_{\hat{\theta}_{f}^{\mathrm{AA}}})$ is further dually, tightly bounded as (cf. (36)):

[TABLE]

As shown, $\Sigma_{\hat{\theta}_{f}^{\mathrm{AA}}}$ can not be upper bounded by the variances alone of the fusion estimators, as it also depends on the discrepancy between the means of two fusing estimators. It is hard to say whether this is an advantage or disadvantage just as whether two distant estimators should be fused or not. Finally, we have the following derivation

[TABLE]

In summary, we have the following remark (cf. Remark 2 for v-fusion). Numerical demonstration will be given in Fig. 5 later on. Inflated variance due to AA or GA has also been pointed out in part by [49, 22], etc.

Remark 5. For Gaussian f-fusion, the AA fusion always leads to a greater variance than the GA fusion does when they use the same fusing weights while the variance of the GA, but not that of the AA, is bounded by the smallest and greatest variances of the fusing estimators.

4.3 MSE Analysis

The MSE of $\hat{\theta}_{f}^{\mathrm{AA}}$ is calculated by (cf. (3))

[TABLE]

which simply indicates that (cf. Remark 4 for v-fusion):

Remark 6. The AA has an MSE that is the linearly weighted average of the MSEs of the fusing estimators and the MSE of the AA is bounded by the smallest and greatest MSEs of the fusing estimators.

Expression (4) provides an easy way to calculate the MSE of $\hat{\theta}_{f}^{\mathrm{GA}}$ based on (34) and (35), i.e.,

[TABLE]

where $a\triangleq\frac{\omega_{1}\Sigma_{1}^{-1}}{\omega_{1}\Sigma_{1}^{-1}+\omega_{2}\Sigma_{2}^{-1}}\in(0,1)$, $b\triangleq\frac{\omega_{2}\Sigma_{2}^{-1}}{\omega_{1}\Sigma_{1}^{-1}+\omega_{2}\Sigma_{2}^{-1}}=1-a\in(0,1)$, $\xi_{1}\triangleq\mu_{1}-\theta$ and $\xi_{2}\triangleq\mu_{2}-\theta$.

It is easy to be verified that \mathrm{mse}_{1}\big{(}\hat{\theta}_{f}^{\mathrm{GA}}\big{)}\geq\mathrm{min}(\Sigma_{1},\Sigma_{2}), \mathrm{mse}_{2}\big{(}\hat{\theta}_{f}^{\mathrm{GA}}\big{)}\geq 0, and so

[TABLE]

where the equation holds when and only when both fusing Gaussian PDFs are unbiased and identical.

We now compare between \mathrm{mse}\big{(}\hat{\theta}_{f}^{\mathrm{GA}}\big{)} and \mathrm{mse}\big{(}\hat{\theta}_{f}^{\mathrm{AA}}\big{)}. First, combining (4.3) and (4) yields

[TABLE]

We have the following simple derivation

[TABLE]

which indicates that (as long as both averaging approaches use the same fusing weights)

[TABLE]

To compare between \mathrm{mse}_{2}\big{(}\hat{\theta}_{f}^{\mathrm{GA}}\big{)} and \mathrm{mse}_{2}\big{(}\hat{\theta}_{f}^{\mathrm{AA}}\big{)}, we consider two specific cases: First, if both fusing Gaussian PDFs are unbiased, i.e., $\mu_{1}=\mu_{2}=\theta$, we have \mathrm{mse}_{2}\big{(}\hat{\theta}_{f}^{\mathrm{GA}}\big{)}=\mathrm{mse}_{2}\big{(}\hat{\theta}_{f}^{\mathrm{AA}}\big{)} and further by using (45),

[TABLE]

where the bounds are approached when the two fusing weights approach [math] and $1$, respectively.

Secondly, if $\Sigma_{1}=\Sigma_{2}$, we have $\omega_{1}=a,\omega_{2}=b$. Subsequently, the following straightforward derivation is obtained

[TABLE]

namely \mathrm{mse}_{2}\big{(}\hat{\theta}_{f}^{\mathrm{GA}}\big{)}\leq\mathrm{mse}_{2}\big{(}\hat{\theta}_{f}^{\mathrm{AA}}\big{)}, as long as they use the same fusing weights. Combining this with (45) yields

[TABLE]

Remark 7. *If both fusing Gaussian PDFs are unbiased or if they have the same variance, the MSE of the GA is smaller than or equals that of the AA and is always greater than the smallest variance of the fusing estimators. *

To gain further insight into their difference in the general case, by means of the Monte Carlo simulation, we consider two Gaussian PDFs $f_{\hat{\theta}_{1}}(x)=\mathcal{N}(x;\mu_{1}=50,100)$ and $f_{\hat{\theta}_{2}}(x)=\mathcal{N}(x;\mu_{2}=60,200)$ and two Gaussian PDFs $f_{\hat{\theta}_{1}}(x)=\mathcal{N}(x;\mu_{1}=50,400)$ and $f_{\hat{\theta}_{2}}(x)=\mathcal{N}(x;\mu_{2}=60,200)$, respectively. The results are shown in Fig. 5 and Fig. 6, respectively, for the real parameter $\theta\in[40,80]$ and fusing weight $\omega_{1}\in(0,1)$. It is seen that (cf. Remark 3 for v-fusion; relevant results can be found in [30])

Remark 8. *For Gaussian f-fusion, the MSE of the AA is in most cases greater than that of the GA, unless $\theta$ is considerably greater than $\mathrm{max}(\mu_{1},\mu_{2})$ and the fusing estimator that has a greater mean has a greater variance. Different from the case of v-fusion, there is no cross-over of their MSEs when the fusing weights change. That is, for certain PDFs and real parameter, one is always better than the other. *

5 PHD Averaging

When multiple objects are involved, the fusing distributions are “multimodal”such as typically a mixture of sub-functions (each of which can be referred to as a “component”) like a GM whose integral is no more (but usually greater than) unit. More precisely, finite set distributions [63, Ch. 5] such as the PHD [38] factories into a cardinality distribution on the number of objects and a localization density conditioned on the cardinality. In this case, while the AA of a sum can be straightforwardly expressed as a cascaded sum of the fusing sums (after re-weighting them) that remains in the same form [42, 43]te, the fractional order exponential power of a sum does not remain as a sum of the same form, and typically approximation must be resorted to; see, e.g., [64, 19, 31, 65, 66].

5.1 Approximate GM-GA Fusion

By omitting the cross-products of different Gaussian functions/components (GCs – we note here that, this simplification only make sense in the case where the GCs in the mixture are well distant), the fractional order exponential power of a mixture of $n$ GCs can be approximated by

[TABLE]

where the covariance inflation of CI, for a weighted Gaussian PDF is equivalent to raising the Gaussian function to a power, which remains Gaussian, namely

[TABLE]

where $\epsilon(\omega,{P})=\sqrt[]{\frac{\det(2\pi{P}\omega^{-1})}{[\det(2\pi{P})]^{\omega}}}=\sqrt[]{(2\pi{P})^{(1-\omega)}\omega^{-1}}$ [14].

In addition, the product of two GCs remains a GC, i.e.,

[TABLE]

where ${P}_{12}=({P}_{1}^{-1}+{P}_{2}^{-1})^{-1},{m}_{12}={P}_{12}({m}_{1}{P}_{1}^{-1}+{m}_{2}{P}_{2}^{-1}),w_{12}=w_{1}w_{2}\mathcal{N}({m}_{1}-{m}_{2};0,{P}_{1}+{P}_{2})$ in which the coefficient $\mathcal{N}({m}_{1}-{m}_{2};0,{P}_{1}+{P}_{2})$ measures the separation of the two GCs.

By using (49), (50) and (51), the approximate GA-fusion of two GMs is ready to be obtained; the interested readers are kindly referred to [31] for the detail. Two points are worth noting. First, the GM-GA fusion requires fusing all pairs of GCs between neighboring sensors, which will result in a multiplied number of GCs. That is, the GA of $J_{1}$ GCs and $J_{2}$ GCs is a mixture of $(J_{1}\cdot J_{2})$ GCs while it is a mixture of $(J_{1}+J_{2})$ GCs in the case of AA fusion. We note here that, a potential means to ameliorate both averaging approaches is to apply mixture merging and pruning to reduce the number of GCs/peaks. Second, to perform the GA fusion as addressed above, the local GM-PHD needs to be normalized to a PDF (cf. (49)). At the end, the resultant GCs need to be properly weighted, such that their sum equals the average of the original weight sums of the fusing GMs. To this end, an extra cardinality consensus scheme may be performed [31, 45]. That is to say, the fusion is performed in different means to the localization density and the cardinality distribution.

5.2 Numerical Comparison between GM-AA and GM-GA

As addressed so far, the AA performs better in v-fusion in the sense of yielding smaller bounds on the variance (cf. Remark 2) and MSE (cf. Remark 3) while the GA performs better in f-fusion in the sense of always yielding smaller variance (cf. Remark 5) and smaller MSE in most cases (cf. Remarks 7 and 8). Therefore, we propose for the PHD fusion a hybrid means by using AA for cardinality fusion [45] and GA for the localization density fusion. This is different from the usual, mere-GA GM fusion [31] , mere-AA GM fusion [42, 43, 46, 44] or mere-cardinality fusion [45]. Further, we note here that, in the case of labelled multi-target density where each label indicates a potential target, the fusion should match/associate the labels between the fusing sources first and then fuses the information (e.g., target existing probability, localization distribution) of the same target, not between different targets [43].

To gain the insight into such a hybrid rules for unnormalized GM fusion, we consider an example in which two GMs are fused in the manner of unweighted AA fusion and unweighted GA fusion, respectively. Here, by unweighted we mean $\omega_{1}=\omega_{2}=\frac{1}{2}$. One GM referred to as GM 1 is given by three GCs (of weight sum 1.8) as follows

[TABLE]

and the other referred to as GM 2 is given by two GCs (of weight sum 1.7) as follows

[TABLE]

As shown, the two GCs $\mathcal{N}(x;10,100)$ and $\mathcal{N}(x;50,100)$ in GM 1 match another two GCs $\mathcal{N}(x;11,100)$ and $\mathcal{N}(x;52,120)$ in GM 2, respectively. They are likely indicating two respective targets. However, there is one extra GC $\mathcal{N}(x;90,200)$ in GM 1, which could be either a false alarm (generated in GM 1) or a real detection (and then there is a misdetection in GM 2) - we hereafter refer to this GC as an isolated GC. The fusion results are given in Fig. 7 in which the fused GM-AA or GM-GA is given in the manner of showing each GC or showing the joint distribution of them, where the joint distribution is superimposition of those of each GC distribution along the state space. We obtain the following two remarks (the first of which is consistent with Remark 5):

Remark 9. The GA fusion generates more significant peaks and lighter tails than the AA fusion does.

Remark 10. *The isolated GC will survive (although its weighted will be reduced) in the AA fusion but will almost vanish in the GA fusion; this indicates that the GA fusion has better capability to suppress false alarm (if the isolated GC is a false alarm in practice) but will also suffer from misdetection (if the isolated GC turns out to be a real detection). This property is a double-edged sword. *

One more comment is in order. As we have addressed earlier in Section 2.3, the support of the AA is the union of those of all initial functions while the support of the GA is the intersection of those of all initial functions. Therefore, assuming that both misdetection and false alarms are independent across fusing GMs, one complete misdetection occurred in one fusing GM (namely the support of the fusing distribution does not really cover the position of the misdetected target) will “dominate” the final GA result (namely the GA must suffer from the misdetection of that corresponding target), no matter how significant the detections are in the other fusing GMs and how many GMs there are. In fact, this problem becomes more serious when more fusing GMs are involved in the GA fusion because a missed detection at any fusing GM can degrade the GA result significantly, and the probability of such a missed detection in the GA obviously becomes larger when more sensors are involved; this may cause the counter-intuitive observation of the GA f-fusion as more fusion leads to worse result [41, 46].

To confirm the above analysis, we consider an example in which six GMs are fused, as shown in Fig. 8. There are five targets in total which lie exactly at position 20, 40, 70, 110 and 200, respectively (as marked by green squares). In each GM, any target is either detected with probability 0.9 and generates a detection (which are given in cyan print) or misdetected with probability 0.1. In addition, false alarms in each GM (which are marked in magenta print) are uniformly distributed in the interval [0, 200] and the number of false alarms is a Poisson random variable with rate 1. Fig. 8 shows the result for one trial based on the given statistics. In the result of the GA fusion, it is seen that the target lying at position 70 was mis-detected while the one lying at 200 was almost mis-detected (as only a low-weighted GC is generated). In the meanwhile, the detections of targets that lied at position 20 and 40 were mixed. These problems can be overcomed in the AA fusion which simply reserves all of the peaks (although this can be another problem).

However, it is worth noting that the GA results we show here might be improved if numerical approximations such as those developed in [64, 19, 65, 66] were used in place of the analytic approximation (49).

Finally, we must stress that it is intractable to exactly compare between the results of the AA and of the GA for the multi-target density/PHD fusion in general due to the two fundamental issues that remain open. The first is regarding target state estimate extraction from the multi-target density/PHD about a random (unknown) number of targets. Two of the most common solutions are referred to as Threshold and Rank rules, respectively. In the former, a threshold is specified in advance and the GCs/peaks whose associated weight is greater than that threshold will all be extracted as estimates while in the latter, the number of estimates is determined firstly and the corresponding number of GCs/peaks of the highest associated weights are extracted as the estimates. Both approaches have their own situation-sensitive strengths as well as deficiencies. The second issue is regarding estimator evaluation metric that has to trade-off between the penalty to misdetection and false alarm and the truth-to-detection distance, which highly depends on the practitioners’ preference.

6 Conclusions

We have analyzed and compared the second order statistics of the GA and of the AA, in terms of averaging random variables (v-fusion) or their PDFs (f-fusion). The key findings that we have obtained can be summarized as follows:

Given that all fusing estimators are unbiased, the AA is always unbiased while the GA may not be, in both v-fusion and f-fusion. Typically, when the fusing PDF is asymmetric, the GA tends to be biased (in the viewpoint of point estimation).

• 2.

For v-fusion,

1. (a)

The variance of both AA and GA can be smaller than the smallest variance of the fusing variables given proper fusing weights, when the fusing variables are little or negatively correlated. 2. (b)

For any two variables, the lowest AA variance (namely the lower bound) that can be yielded by adjusting the fusing weights is smaller than that of the GA variance. 3. (c)

The lowest bound of the MSE of either the AA or the GA is their corresponding variance, which is only possible when the real parameter lies between two variables and proper fusing weights are used.

• 3.

For Gaussian f-fusion,

1. (a)

The AA fusion always leads to a greater variance than the GA fusion does, for using the same fusion weights. 2. (b)

The AA has an MSE that is the weighted average of the MSEs of the fusing estimators (and so it is bounded by the smallest and greatest MSEs). 3. (c)

The GA fusion tends to perform better than the AA fusion in obtaining smaller MSE in most cases.

• 4.

For PHD-fusion based on a hybrid use of GA for distribution fusion and AA for cardinality fusion,

1. (a)

The GA fusion generates more significant peaks and lighter tails than the AA fusion does; in order words, the GA is comparably more accurate but less robust. 2. (b)

The GA fusion has better capability to suppress false alarm but also suffers from higher risk in causing misdetection as compared to the AA fusion, especially in the case of a large number of fusing sources.

Appendix A: Lower Bound of $h(w)$ as in (18)

Here, we analyze the lower bound of function $h(w)$ as given in (18) for $w\in(0,1)$ and $\alpha\geq 1,\rho\in(-1,1)$. Strightforwardly, the derivative of $h(w)$ with respect to $w$ is

[TABLE]

Setting it to zero yields

[TABLE]

This, however, may not satisfy the rule that $0<w<1$ and if not, cannot be used. We discuss two opposite cases:

1. When $\rho<\alpha^{-\frac{1}{2}}$:

In this case, (53) satisfies $0<w<1$ and yields

[TABLE]

Furthermore, by applying $\rho<\alpha^{-\frac{1}{2}}$, we obtain $\frac{\mathrm{d}^{2}h(w)}{\mathrm{d}w^{2}}=2(\alpha+1-2\rho\alpha^{\frac{1}{2}})>0$. This indicates that the bound given in (54) is indeed the lower bound. That is, if $\rho<\alpha^{-\frac{1}{2}}$, the optimal fusing weights to get the minimal $h(w)$ are given by

[TABLE]

2. When $\rho\geq\alpha^{-\frac{1}{2}}$:

In this case, $w<0$ and so, (53) can not be used. Considering that $h(w)$ is a convex function of $w$ and $\frac{\mathrm{d}^{2}h(w)}{\mathrm{d}w^{2}}>0$, we obtain dual bounds of $h(w)$ at the boundaries of the support interval of the fusing weights, namely

[TABLE]

Appendix B: An example for averaging two functions

Supposing the real parameter $\theta=\frac{2\hskip 1.42262pt\sqrt[]{2}}{3}$, an unbiased estimator is given with a uniform PDF on the interval $(0,1]$, namely

[TABLE]

and another unbiased estimator is given with asymmetric PDF

[TABLE]

Simply for $\omega_{1}=\omega_{2}=\frac{1}{2}$, the AA remains unbiased as (30) holds but the GA (10) is

[TABLE]

Then, (31) reads

[TABLE]

Acknowledgement

This work was supported in part by the Marie Skłodowska-Curie Individual Fellowship under Grant 709267, in part by the Northwestern Polytechnical University and in part by the MOVIURBAN Project (Ref. SA070U 16) co-financed with Junta de Castilla y León, Consejería de Educacin and FEDER funds.