Visual Multiple-Object Tracking for Unknown Clutter Rate

TL;DR

This paper introduces a multi-object tracking algorithm capable of handling unknown false measurement rates by combining robust multi-Bernoulli filtering for clutter estimation with generalized labeled multi-Bernoulli filtering for target tracking, validated on real videos.

Contribution

The paper presents a novel multi-object tracking method that effectively manages unknown clutter rates using combined filtering techniques, enhancing real-world tracking reliability.

Findings

Effective clutter estimation in real-world scenarios

Improved tracking accuracy with unknown false measurement rates

Validated performance on real video data

Abstract

In multi-object tracking applications, model parameter tuning is a prerequisite for reliable performance. In particular, it is difficult to know statistics of false measurements due to various sensing conditions and changes in the field of views. In this paper we are interested in designing a multi-object tracking algorithm that handles unknown false measurement rate. Recently proposed robust multi-Bernoulli filter is employed for clutter estimation while generalized labeled multi-Bernoulli filter is considered for target tracking. Performance evaluation with real videos demonstrates the effectiveness of the tracking algorithm for real-world scenarios.

Figures

Tables

Table 1. TABLE I: Comparison with the state-of-the-art trackers

Dataset	Method	Recall	Precision	FPF	GT	MT	PT	ML	Frag	IDS
	Boosted GLMB	90.2 %	89.5 %	0.03	19	90 %	10 %	0.0 %	23	10
PETS09-S2L1	GLMB [19]	82.6 %	81.4 %	0.16	19	82.7 %	17.3 %	0.0 %	23	12
	RMOT [15]	80.6 %	85.4 %	0.25	19	84.7 %	15.3 %	0.0 %	20	11
	Boosted GLMB	83.4 %	85.6 %	0.10	10	80 %	20 %	0.0 %	12	16
TUD-Stadtmitte	GLMB [19]	80.0 %	83.0 %	0.16	10	78.0 %	22.0 %	0.0 %	23	12
	RMOT [15]	82.9 %	86.6 %	0.19	10	80 %	20 %	0.0 %	10	16
ETH	Boosted GLMB	73.1 %	82.6 %	0.78	124	60.4 %	34.6 %	5.0 %	110	20
BAHNHOF and	GLMB [19]	71.5 %	76.3 %	0.88	124	58.7 %	27.4 %	13.9 %	112	40
SUNNYDAY	RMOT [15]	71.5 %	76.3 %	0.98	124	57.7 %	37.4 %	4.8 %	68	40

Table 2. TABLE II: Comparison of averaged OSPA distance

Method	Average OSPA
Boosted GLMB (Ours)	5
MHT [9]	8.5

Equations

$$\breve{\mathbb{X}}=\mathbb{X}\times \{0,1\}$$

$$\breve{X_k}=\{{\breve{x}}_{k,i}{\}}_{i=1}^{T_k}\subset \breve{\mathbb{X}},Z_k=\{z_{k,j}{\}}_{j=1}^{O_k}\subset \mathbb{Z},$$

$$\begin{array}[]{llll}\breve{X}_{k}\displaystyle{=\bigcup_{\breve{x}_{k-1}\in\breve{X}_{k-1}}{S_{k|k-1}(\breve{x}_{k-1})}~{}~{}\bigcup{\Gamma_{k}}},\end{array}$$

$$\begin{array}[]{llll}Z_{k}{=Z_{T,0,k}\bigcup Z_{T,1,k}},\end{array}$$

$$\{(r_{k-1}^{(i)},p_{u,k-1}^{(i)}){\}}_{i=1}^{M_{k-1}}.$$

$$\{(r_{k|k-1}^{(i)},p_{u,k|k-1}^{(i)}){\}}_{i=1}^{M_{k|k-1}}.$$

$$\begin{array}[]{ll}r_{P,k|k-1}^{(i)}=r_{k-1}^{(i)}\sum_{u=0,1}\langle p_{u,k-1}^{(i)},p_{S,u,k}\rangle,&\\ p_{u,k|k-1}^{(i)}(x)=\frac{\left\langle f_{u,k|k-1}(x|\cdot,)p_{u,k-1}^{(i)},p_{S,u,k}\right\rangle}{\left\langle p_{u,k-1}^{(i)},p_{S,u,k}\right\rangle},&\end{array}$$

$$\begin{array}[]{ll}\{(r_{L,k}^{(i)},p_{L,u,k}^{(i)})\}_{i=1}^{M_{k|k-1}}\cup\{(r_{U,k}(z),p_{U,u,k}(\cdot;z))\}_{z\in Z_{k}}.&\end{array}$$

$$\begin{array}[]{lllll}r_{L,k}^{(i)}=\sum_{u=0,1}r_{L,u,k}^{(i)},&&&&\\ r_{L,u,k}^{(i)}=\frac{r_{k|k-1}^{(i)}\left\langle p_{u,k|k-1}^{(i)},1-p_{D,u,k}\right\rangle}{1-r_{k|k-1}^{(i)}\sum_{u^{\prime}=0,1}\left\langle p_{u^{\prime},k|k-1}^{(i)},p_{D,u^{\prime},k}\right\rangle},&&&&\\ p_{L,u,k}^{(i)}(x)=\frac{\big{(}1-p_{D,u,k}\big{)}p_{u,k|k-1}^{(i)}(x)}{\sum_{u^{\prime}=0,1}\left\langle p_{u^{\prime},k|k-1}^{(i)},1-p_{D,u^{\prime},k}\right\rangle},&&&&\\ r_{U,k}(z)=\sum_{u=0,1}r_{U,u,k}(z),&&&&\\ r_{U,u,k}(z)=\frac{\sum_{i=1}^{M_{k|k-1}}\frac{r_{k|k-1}^{(i)}\big{(}1-r_{k|k-1}^{(i)}\big{)}\left\langle p_{u,k|k-1}^{(i)},g_{u,k}(z,|\cdot)p_{D,u,k}\right\rangle}{\big{(}1-r_{k|k-1}^{(i)}\sum_{u^{\prime}=0,1}\left\langle p_{u^{\prime}=0,1}^{(i)},p_{D,u^{\prime},k}\right\rangle\big{)}^{2}}}{\sum_{i=1}^{M_{k|k-1}}\frac{r_{k|k-1}^{(i)}\sum_{u^{\prime}=0,1}\left\langle p_{u,k|k-1}^{(i)},g_{u^{\prime},k}(z|\cdot)p_{D,u^{\prime},k}\right\rangle}{1-r_{k|k-1}^{(i)}\sum_{u^{\prime}=0,1}\left\langle p_{u^{\prime},k|k-1}^{(i)},p_{D,u^{\prime},k}\right\rangle}},&&&&\\ p_{U,u,k}(x;z)=&&&&\\ \frac{\sum_{i=1}^{M_{k|k-1}}\frac{r_{k|k-1}^{(i)}}{1-r_{k|k-1}^{(i)}}p_{u,k|k-1}^{(i)}(x)g_{u,k}(z|x)\cdot p_{D,u,k}}{\sum_{u^{\prime}=0,1}\sum_{i=1}^{M_{k|k-1}}\frac{r_{k|k-1}^{(i)}}{1-r_{k|k-1}^{(i)}}\left\langle p_{u^{\prime},k|k-1}^{(i)},g_{u^{\prime},k}(z|\cdot)p_{D,u^{\prime},k}\right\rangle}\end{array}$$

$${\mathbf{X}}_k=\{{\mathbf{x}}_{k,1},...,{\mathbf{x}}_{k,N_k}\}\in \mathcal{F}({\mathbb{X}\times \mathbb{L}}_k),$$

$${\pi }_{k-1}(\mathbf{X})=\Delta (\mathbf{X})\sum _{c\in \mathbb{C}}{\omega }_{k-1}^{(c)}({\mathcal{L}}_{\mathbf{X}}){\left[p_{k-1}^{(\xi )}\right]}^{\mathbf{X}}.$$

$${\pi }_{k|k-1}(\mathbf{X})=\Delta (\mathbf{X})\sum _{c\in \mathbb{C}}{\omega }_{k|k-1}^{(c)}({\mathcal{L}}_{\mathbf{X}}){\left[p_{k|k-1}^{(c)}\right]}^{\mathbf{X}}$$

$$\begin{array}[]{ll}\leavevmode\resizebox{422.77661pt}{}{$\mathbf{\pi}_{k|k}(\mathbf{X}|Z_{k})=\Delta(\mathbf{X})\displaystyle\sum_{c\in\mathbb{C}}\sum\limits_{\theta\in\Theta_{k}}\omega_{Z_{k}}^{(c,\theta)}(\mathcal{L}_{\mathbf{X}})\left[p_{k|k}^{(c,\theta)}(\cdot|Z_{k})\right]^{\mathbf{X}}$}\end{array}$$

$$\widehat{{\lambda }_c}=\sum _{i=1}^{M_k}{r}_{0,k}^{(i)}{p}_{D,0,k}$$

$$f_{1,k|k-1}=\mathcal{N}(x_k;m_{x,1,k|k-1}(x_{k-1}),P_{x,1,k|k-1}),$$

$$F(\omega )=\left[\begin{array}{cccc}1& \frac{\text{sin}\omega T}{\omega }& 0& -\frac{1-\text{cos}\omega T}{\omega }\\ 0& \text{cos}\omega T& 0& -\text{sin}\omega T\\ 0& \frac{1-\text{cos}\omega T}{\omega }& 1& \frac{\text{sin}\omega T}{\omega }\\ 0& \text{sin}\omega T& 0& \text{cos}\omega T\end{array}\right],G=\left[\begin{array}{cc}\frac{T^2}{2}& 0\\ T& 0\\ 0& \frac{T^2}{2}\\ 0& T\end{array}\right],$$

$$\begin{array}[]{llll}g_{k}(z_{k}|x_{k})=\mathcal{N}(z_{k};m_{z,1,k}(x_{k}),P_{z,1,k}),\end{array}$$

$$\begin{array}[]{llll}g_{k}(z_{k}|x_{k})=\mathcal{N}(z_{k};Hx_{k},\Sigma),\end{array}$$

Full text

I Introduction

Multi-object tracking is one of fundamental problems in many applications. There are abundant research works, however, it is still far from practical use. The overwhelming majority of multi-target tracking algorithms are built on the assumption that multi-object system model parameters are known a priori, which is generally not the case in practice [1], [4]. While tracking performance is generally tolerant to mismatches in the dynamic and measurement noise, the same cannot be said about missed detections and false detections. In particular, mismatches in the specification of missed detection and false detection model parameters such as detection profile and clutter intensity can lead to a significant bias or even erroneous estimates [25].

Unfortunately, except for a few application areas, exact knowledge of model parameters is not available. This is especially true in visual tracking, in which the missed detection and false detection processes vary with the detection methods. The detection profile and clutter intensity are obtained by trial and error. A major problem is the time-varying nature of the missed detection and false detection processes. Consequently, there is no guarantee that the model parameters chosen from training data will be sufficient for the multi-object filter at subsequent frames.

In radar target tracking applications, stochastic multi-object tracking algorithms based on Kalman filtering or Sequential Monte Carlo (SMC) method have been widely used [4], [16]. This approach also has been used in visual multi-object tracking research [7], [8], [23]. On the other hand, deterministic approach such as network flow [5], continuous energy optimisation [10], has become a popular method for multi-object tracking problem in visual tracking application. This approach is known to be free from tuning parameters, however, it is useful only when reliable object detection is available.

Unknown observation model parameters (i.e., clutter rate, detection profile) in online multi-object filtering was recently formulated in a joint estimation framework using random finite set (RFS) approach [2], [3]. Recently, Mahler [25] showed that clever use of the CPHD filter can accommodate unknown clutter rate and detection profile. In [26] it was demonstrated that by bootstrapping clutter estimator from [25] to the Gaussian mixture CPHD filter [17], performed very close to the case with known clutter parameter can be achieved. [27] extended it to multi-Bernoulli filter with SMC implementation. The multi-Bernoulli filter was used for visual multi-object tracking in [11]. While the solution for filtering with unknown clutter rate exists, these filters do not provide tracks that identify different objects. In particular, the conference version of this work [11] is seriously extended as a new algorithm that is able to provides track identities with completely new structure and evaluated using challenging pedestrian tracking and cell migration experiments To the best of our knowledge this paper is the first attempt for handling unknown false measurement information in online tracking. The main contribution of this paper is to design a multi-object tracker that also produces trajectories and estimates unknown clutter rate on the fly.

II Problem Formulation

Let ${\mathbb{X}}=\mathbb{R}^{n_{x}}$ denote the space of the target kinematic state, and $\{0,1\}$ denote the discrete space of labels for clutter model and actual targets. Then, the augmented state space is given by

[TABLE]

where $\times$ denotes a Cartesian product. Consequently, the state variable contains the kinematic state, and target/clutter indicator. We follow the convention from [27] that the label $u=0$ will be used as a subscript to denote the clutter generators and the label $u=1$ for actual targets.

Suppose that there are $T_{k}$ target and clutter object, and we have $O_{k}$ observations (i.e., detections). In the RFS framework, the collections of targets (including clutter objects) and measurements can be described as finite subsets of the state and observation spaces, respectively as (2)

[TABLE]

where $\breve{x}_{k,i}$ represent either the kinematic state of actual target or clutter target; $z_{k,j}$ is a measurement, and $\mathbb{Z}$ is the space of measurement, respectively. Considering the dynamic of the state, the RFS model of the multi-target state at time $k$ consists of surviving targets and new targets entering in the scene. This new set is represented as the union

[TABLE]

where ${\small{\Gamma_{k}}}$ is a set of spontaneous birth objects (actual target or clutter targets) and ${\small{S_{k|k-1}(\cdot)}}$ is the set of survived object states at time $k$ with survival probability $p_{S}(x)<1$.

The set of observations given the multi-target state is expressed as

[TABLE]

where $Z_{T,0,k}$ and $Z_{T,1,k}$ are, respectively, sets of clutter and target-originated observations with unknown detection probability $p_{D}(x)<1$.

With the RFS multi-target dynamic and measurement model, the multi-object filtering problem amounts to propagating multi-target posterior density recursively forward in time via the Bayes recursion. Note that in the classical solution to this filtering problem such as PHD [2], CPHD [3], and multi-Bernoulli filters [18], [22], [23], instead of clutter target measurement set, the Poisson clutter intensity is given and the detection profile $p_{D}(x)$ is also known a priori [1].

III Multi-object tracker with unknown clutter rate

The aim of this paper is to propose a new online multi-object tracker that is able to accommodate unknown clutter rate. For this purpose, the Robust Multi-Bernoulli (RMB) filter [27] is employed to adapt unknown clutter rate. Then, the estimated clutter rate is plugged into the Generalized Multi-Bernoulli (GLMB) tracker [19] to boost the performance in real-world scenarios.

III-A Robust Multi-Bernoulli Filter

The multi- Bernoulli filter parametrizes the multi-object posterior density by using a set of pair, i.e., Bernoulli parameter, $\{(r^{(i)},p^{(i))}\}_{i=1}^{M}$ where $r^{(i)}$ and $p^{(i)}$ represent the existence probability and the density of the state among $M$ Bernoulli components. In the following the predicted and updated densities are represented by propagating a set of Bernoulli parameters. The multi-Bernoulli filter recursion for extended state space called RMB filter [27] is summarized to make the paper self-contained.

If the posterior multi-object density of the multi-Bernoulli form at time $k-1$ is given as

[TABLE]

Then, the predicted intensity is approximated by the following multi-Bernoulli

[TABLE]

A set of predicted Bernoulli components is a union of birth components $\{(r_{\Gamma,k}^{(i)},p_{\Gamma,u,k}^{(i)})\}_{i=1}^{M_{\Gamma,k}}$ and surviving components $\{(r_{k|k-1}^{(i)},p_{u,k|k-1}^{(i)})\}_{i=1}^{M_{k|k-1}}$. The birth Bernoulli components are chosen a priori by considering the entrance region of the visual scene, e.g., image border. The surviving components are calculated by

[TABLE]

where $x$ is the kinematic state, $p_{S,u,k}$ is the survival probability to time $k$ and $f_{u,k|k-1}(x|\cdot)$ is the state transition density specified by either for actual target $f_{1,k|k-1}(x|\cdot)$ or for clutter target $f_{0,k|k-1}(x|\cdot)$.

If at time $k$, the predicted multi-target density is multi-Bernoulli of the form (6), then the updated multi-Bernoulli density approximation is composed of the legacy components with the subscript $L$ and the measurement updated components with the subscript $U$ as follows (8)

[TABLE]

The legacy and measurements updated components are calculated by a series of equations (9) as follows.

[TABLE]

where $p_{D,u,k}$ is the state dependent detection probability, $g_{u,k}(z|x)$ is the measurement likelihood function that will be defined in the following section. Note that the SMC implementation of summarized equations (6)-(10) can be found in [27].

III-B Boosted Generalized labeled Multi-Bernoulli Filter

The generalized labeled multi-Bernoulli (GLMB) filter provides a solution of multi-object Bayes filter with unique labels. In this paper, the GLMB filter is used as a tracker that returns trajectories of multi-object given the estimated clutter rate from the RMB. As shown in Fig. 1, GLMB and RMB filters are interconnected by sharing tracking parameters and facilitate feedback mechanism in order for robust tracking against time-varying clutter background. Note that one step RMB filter is used for the estimation of clutter rate, thus, it is not a parallel implementation of independent filter.

We call the proposed tracker as Boosted GLMB tracker.

In multi-object tracking with labels, formally, the state of an object at time $k$ is defined as $\mathbf{x}_{k}=(x_{k},\ell_{k})\in\mathbb{X\times L}_{k}$, where $\mathbb{L}_{k}$ denotes the label space for objects at time $k$ (including those born prior to $k$). Note that $\mathbb{L}_{k}$ is given by $\mathbb{B}_{k}\cup\mathbb{L}_{k-1}$, where $\mathbb{B}_{k}$ denotes the label space for objects born at time $k$ (and is disjoint from $\mathbb{L}_{k-1}$) and we do not consider clutter generator in designing GLMB, thus, the label $u$ is omitted. Suppose that there are $N_{k}$ objects at time $k$ as (2), but only consider actual target with label $\mathbf{x}_{k,1},...,\mathbf{x}_{k,N_{k}}$, in the context of multi-object tracking,

[TABLE]

where $\mathcal{F}(\mathbb{X\times L}_{k})$ denotes the space of finite subsets of $\mathbb{X\times L}_{k}$. We denote cardinality (number of elements) of $\mathbf{X}$ by $|\mathbf{X}|$ and the set of labels of $\mathbf{X}$, $\{\ell:(x,\ell)\in\mathbf{X}\}$, by $\mathcal{L}_{\mathbf{X}}$. Note that since the label is unique, no two objects have the same label, i.e. $\delta_{|\mathbf{X}|}(|\mathcal{L}_{\mathbf{X}}|)=1$. Hence $\Delta(\mathbf{X})\triangleq$ $\delta_{|\mathbf{X}|}(|\mathcal{L}_{\mathbf{X}}|$ is called the distinct label indicator.

In the GLMB the posterior density takes the form of a generalized labeled multi-Bernoulli

[TABLE]

Given the posterior multi-object density of the form (11), the predicted multi-object density to time $k$ is given by

[TABLE]

where

$\omega_{k|k-1}^{(c)}(L)=w_{B,k}(L\cap\mathbb{B}_{k})\omega_{S,k}^{(c)}(L\cap\mathbb{L}_{k-1}),$

$p_{k|k-1}^{(c)}(x,\ell)=1_{\mathbb{L}_{k-1}}(\ell)p_{S,k}^{(c)}(x,\ell)+(1-1_{\mathbb{L}_{k-1}}(\ell))p_{B,k}(x,\ell),$

$p_{S,k}^{(c)}(x,\ell)=\frac{\left\langle p_{S,k-1}(\cdot,\ell)f_{k|k-1}(x|\cdot,\ell),p_{k-1}^{(c)}(\cdot,\ell)\right\rangle}{\eta_{S}^{(c)}(\ell)},$

$\eta_{S,k}^{(c)}(\ell)=\int\left\langle p_{S,k-1}(\cdot,\ell)f_{k|k-1}(x|\cdot,\ell),p_{k-1}^{(c)}(\cdot,\ell)\right\rangle dx,$

   $\omega_{S,k}^{(c)}(J)=[\eta_{S,k}^{(c)}]^{L_{k-1}}\sum_{I\subseteq\mathbb{L}_{k-1}}1_{I}(J)[q_{S}^{(c)}]^{I-J}\omega_{k-1}^{(c)},$

$q_{S}^{(c)}(\ell)=\left\langle 1-p_{S,k-1}(\cdot,\ell),p_{k-1}^{(c)}(\cdot,\ell)\right\rangle$,

where $c$ is the index for track hypothesis, $L$ is an instance of label set, $I$ is track labels from previous time step.

Moreover, the updated multi-object density is given by

[TABLE]

where $\Theta_{k}$ is the space of mappings $\theta:\mathbb{L}_{k}\rightarrow\{0,1,...,|Z_{k}|\},$ such that $\theta(i)=\theta(i^{\prime})>0~{}$implies$~{}i=i^{\prime}$, and

 $\omega_{Z_{k}}^{(c,\theta)}(L)\propto\delta_{\theta^{-1}(\{0:|Z_{k}|\})}(L)\omega_{k|k-1}^{(c)}(L)[\eta_{Z_{k}}^{(c,\theta)}]^{L},$

$p_{k|k}^{(c,\theta)}(x,\ell|Z_{k})=\frac{p_{k|k-1}^{(c)}(x,\ell)\psi_{Z_{k}}(x,\ell;\theta)}{\eta_{Z_{k}}^{(c,\theta)}(\ell)},$

$\eta_{Z_{k}}^{(c,\theta)}(\ell)=\left\langle p_{k|k-1}^{(c)}(\cdot,\ell),\psi_{Z_{k}}(\cdot,\ell;\theta)\right\rangle,$

$\psi_{Z_{k}}(x,\ell;\theta)=\delta_{0}(\theta(\ell))(1-p_{D,k}(x,\ell))\\ \indent\indent\indent\indent\indent+(1-\delta_{0}(\theta(\ell)))\frac{p_{D,k}(x,\ell)g_{k}(z_{\theta(\ell)}|x,\ell)}{\kappa_{k}(z_{\theta(\ell)})}$

where $\kappa_{k}\sim\hat{\lambda_{c}}\mathcal{U(\mathcal{Z})}$ denotes the clutter density. $\hat{\lambda_{c}}$ is the estimated clutter rate from the RMB filter. Specifically, the extraction of clutter rate can be simply obtained by the EAP estimate of clutter target number as

[TABLE]

where $r^{(i)}_{0,k}$ is the existence probability of clutter target introduced in the previous section, $p_{D,0,k}$ is the probability of detection for clutter targets, $\mathcal{U}(\mathcal{Z})$ is a uniform density on the observation region $\mathcal{Z}$.

IV Experimental results

In this section, two types of experimental results are given. A nonlinear multi-object tracking example is tested in order to show the performance of the proposed tracker with respect to the standard performance metric, i.e., OSPA distance [28]. In addition, the proposed tracker is also evaluated for visual multi-object tracking datasets [13], [14], [31].

IV-A Object motion model and basic parameters

The target dynamic described as a coordinated turn model as (15)

[TABLE]

where $m_{x,1,k|k-1}(x_{k-1})=[F(\omega_{k-1})x_{k-1},\omega_{k-1}]^{T}$, $P_{x,1,k|k-1}=diag([\sigma^{2}_{w}GG^{T},\sigma^{2}_{\omega}])$,

[TABLE]

where $T$ is the sampling time, $\sigma_{w}$ is the standard deviation of the process noise, $\sigma_{\omega}$ is the standard deviation of the turn rate noise. These standard deviation values are determined by the maximum allowable object motion with respect to the image frame rate. For clutter targets, the transition density $f_{0,k|k-1}$, is given as a random walk to describe arbitrary motion [27].

IV-B Numerical example

The proposed algorithm is tested with a nonlinear multi-target tracking scenario in [19], [27]. The actual target is observed from noisy bearing and range information $z_{k}=[\theta_{k},r_{k}]^{T}$ and its likelihood function is given by

[TABLE]

where $m_{z,1,k}(x_{k})=[\text{arctan}(p_{x,k}/p_{y,k}),\sqrt{p^{2}_{x,k}+p^{2}_{y,k}}]$ and $P_{z,1,k}=diag([\sigma^{2}_{\theta},\sigma^{2}_{r}])$. For RMB implementation, we follow the same parameter setting as given in [27]. Performance comparison between the GLMB tracker with known clutter rate and the proposed tracker (Boosted GLMB tracker) is studied. As can be seen in Fig. 1, OSPA distances for both trackers are similar. This result verifies that the Boosted GLMB shows reliable performance even when the clutter rate is unknown.

IV-C Pedestrian tracking in vision

For the evaluation in real-world data, we are interested in tracking of multiple pedestrians. To detect pedestrians, we apply the state-of-the-art pedestrian detector proposed by Piotr et. al, called ACF detector [29]. The detector used in the experiment integrates a set of image channels (normalized gradient magnitude, histogram of oriented gradients, and LUV color channels) to extract various types of features in order to discriminate objects from the background.

Assuming that the object state $x_{k}=[p_{x,k}\dot{p}_{x,k},p_{y,k}\dot{p}_{y,k}]^{T}$ (x-, y- positions and velocities) is observed with additive Gaussian noise, the measurement likelihood function is given by

[TABLE]

where $\mathcal{N}(z;m,P)$ denotes a normal distribution with mean $m$ and covariance $P$, $z_{k}$ is the response from designated detector; $H=[1~{}~{}0~{}~{}0~{}~{}0;~{}~{}1~{}~{}0~{}~{}0~{}~{}0]$ i,e., x-, y- position is observed by the detector, $\Sigma$ is the covariance matrix of the observation noise.

Sample detection results in Fig. 2 contain false positive detections from other types of object with similar shapes as pedestrians. Based on our experiences ACF detector is robust to partial occlusions, however, there are more false positive detections than other single-model based detectors [30]. Thus, it is relatively difficult to remove false positive detections by hard thresholding when it is used for visual scenes with time-varying imaging conditions or moving camera. In particular, in visual scene from autonomous vehicles, the average number of clutters (i.e., clutter rate) is varying with respect to the change in the field of view due to the vehicle pose change.

The basic assumption behind the existing visual multi-object tracking is that the offline-designed detector, e.g., HOG detector [29], [30] gives reasonably clean detections. Thus, direct data association algorithms such as [5], and [10] show reasonable performance with minor number of clutter measurements. However, in practice, there are false positive detections which make data association results inaccurate and computationally intensive.

”S2.L1” sequence from the popular PETS’09 dataset [31], ”TUD-Stadtmitte” sequence from TUD dataset [13], and ”Bahnhof” and ”Sunnyday” sequences from ETHZ dataset [14] are tested in the experiment where maximum 8-15 targets are moving in the scene. The number of targets varies in time due to births and deaths, and the measurement set includes target-originated detections and clutter. In our experiments, unlike previous works, we use the ACF detector with low threshold for nonmaximum suppression so as to have less number of miss-detections but increased false alarms with time-varying rate. It is more realistic setting especially in ETHZ dataset that is recorded with frequent camera view changes. The Boosted GLMB, is compared with the original GLMB (with fixed clutter rate) [19], and state-of-the-art online Bayesian multi-object tracker called RMOT [15]. Quantitative experiment results are summarized in Table 1 using well-known performance indexes given in [6]. In Table 1, Boosted GLMB shows superior performance compared to the GLMB in all indexes and compatible with the recent online tracker, RMOT. The proposed Boosted GLMB outperforms other trackers with respect to FPF where tracker is able to effectively reject clutters with estimated clutter rate. On the hand, inferior performance in Fragmentation and ID switches are observed compared with RMOT because of the lack of relative motion model.

In summary, it is verified from the experiment that the Boosted GLMB filter is effective when the clutter rate is not known a priori which is often required in real-world applications. We make experiments using unoptimized MATLAB code with Intel 2.53GHz CPU laptop. The computation time per one image frame of size 768$\times$586 is 0.2s which is reasonably suitable for real-time visual tracking application. Further improvements can be made by code optimization to speed up.

IV-D Cell migration analysis in microscopy image

The proposed algorithm is also tested with live-cell microscopy image data for cell migration analysis. The proposed GLMB tracking method is tested on a real stem cell migration sequence as illustrated in Fig. 4. The image sequence is recorded for 3 days, i.e., 4320 min and each image is taken in every 16 min.

Performance comparison is conducted with the state-of-the-art Multiple Hypothesis Tracker (MHT) [9]. The same motion and measurement models are used as in the first experiments and spot detection in [9] is applied for the fair comparison. As shown in Fig. 5, the Boosted GLMB provides reliable tracking results compared to the MHT. The MHT is tuned to obtain the best tracking results. The Boosted GLMB tracker produces significantly less false tracks and alleviate fragmented tracks because the tracker efficiently manages time-varying clutter information and keep confident tracks. Quantitatively, as can be seen in Table II, time averaged OSPA distances [28] for both trackers verify that the Boosted GLMB shows reliable performance even when the clutter rate is unknown.

V Conclusion

In this paper, we propose a new multi-object tracking algorithm for unknown clutter rate based on two interconnected random finite set filters. Unknown clutter rate is estimated using one step robust Bernoulli filter [27]. Then, trajectories of objects are estimated using [19] with estimated clutter rate online. Two filters are sharing tracking parameters so that there is no need for tuning of clutter parameters. Comparison results via a synthesized nonlinear multi-object tracking and visual tracking datasets (visual surveillance and biomedical) with state-of-the-art online trackers illustrate that the proposed multi-object tracker shows reliable performance. Interesting future research direction would be the extension of the tracking algorithm to adaptive survival probability and handling of missed-detections for further improvement.