Path Assignment Techniques For Vehicle Tracking

TL;DR

This paper reviews vehicle path estimation techniques and introduces two new methods that utilize posterior probability distributions and median estimators to improve lane assignment accuracy in driver assistance systems.

Contribution

The paper proposes two novel path assignment methods that delay filtering to reduce artifacts and improve lane detection accuracy in vehicle tracking.

Findings

Methods outperform traditional approaches in ROC tests

Filtering later in processing reduces delays and artifacts

Experimental data validates the effectiveness of proposed techniques

Abstract

Many driver assistance systems such as Adaptive Cruise Control require the identification of the closest vehicle that is in the host vehicle's path. This entails an assignment of detected vehicles to the host vehicle path or neighboring paths. After reviewing approaches to the estimation of the host vehicle path and lane assignment techniques we introduce two methods that are motivated by the rationale to filter measured data as late in the processing stages as possible in order to avoid delays and other artifacts of intermediate filters. These filters generate discrete posterior probability distributions from which a path or "lane" index is extracted by a median estimator. The relative performance of those methods is illustrated by a ROC using experimental data and labeled ground truth data.

Tables

Table 1. TABLE I: Path index

$l$	interpretation
$0$	left of left path
$1$	left path
$2$	center (host) path
$3$	right path
$4$	right of right path

Equations

$$z_{P,k}=\{y_{P,k}^{obj},y_{P,k}^{bnd,1},y_{P,k}^{bnd,2},y_{P,k}^{bnd,3},y_{P,k}^{bnd,4}\}.$$

$$p(l_k|Z_k)=\frac{p(z_{P,k}|l_k)p(l_k|Z_{k-1})}{{\sum }_{m_k\in L}p(z_{P,k}|m_k)p(m_k|Z_{k-1})}$$

$$p(z_{P,k}|l_k)=p(l_k|z_{P,k})\frac{p(z_{P,k})}{p(l_k)}$$

$$p(l_k|Z_k)=\frac{p(l_k|z_{P,k})p(l_k|Z_{k-1})}{{\sum }_{m_k\in L}p(m_k|z_{P,k})p(m_k|Z_{k-1})}$$

$$p(l_k|z_{P,k})$$

$$p({\tilde{y}}_{P,k}^{bnd,l}|z_{P,k})=$$

$$\mathcal{N}({\tilde{y}}_{P,k}^{bnd,l};{\mu }_{P,k}^{obj}-{\mu }_{P,k}^{bnd,l},\sqrt{({\sigma }_{P,k}^{obj}{)}^2+({\sigma }_{P,k}^{bnd,l}{)}^2})$$

$$p(l_k|z_{P,k})$$

$$p(l_k|Z_{k-1})=\sum _{l_{k-1}}p(l_k|l_{k-1})p(l_{k-1}|Z_{k-1})$$

$$p(l_k|l_{k-1})=$$

$$\left(\begin{array}{ccccc}1-ϵ-\eta & ϵ+0.5|\eta |-\eta & 0& 0& 0\\ ϵ+\eta & 1-2ϵ-|\eta |& ϵ+0.5|\eta |-\eta & 0& 0\\ 0& ϵ+0.5|\eta |+\eta & 1-2ϵ-|\eta |& ϵ+0.5|\eta |-\eta & 0\\ 0& 0& ϵ+0.5|\eta |+\eta & 1-2ϵ-|\eta |& ϵ-\eta \\ 0& 0& 0& ϵ+0.5|\eta |+\eta & 1-ϵ+\eta \end{array}\right)$$

$$\sum _{l_k\le l_k^{est}}p(l_k|Z_k)\ge 0.5\text{ and }\sum _{l_k\ge l_k^{est}}p(l_k|Z_k)\ge 0.5$$

$${\xi }_{k+1}={\xi }_k+\Delta t_k{u}_k+\Delta t_k{\nu }_k$$

$$z_k={\xi }_k+w_k$$

$$\left(\begin{array}{c}{x}_p^{center}\\ y_p^{center}\end{array}\right)=\left(\begin{array}{cc}\cos (\alpha )& -\sin (\alpha )\\ \sin (\alpha )& \cos (\alpha )\end{array}\right)\left(\begin{array}{c}0\\ r\end{array}\right)$$

$$y_P=r-\mathrm{sgn}(r)\sqrt{{\left(x+\sin (\alpha )r\right)}^2+{\left(y-\cos (\alpha )r\right)}^2}$$

$${\mu }_{P,k}$$

$${\sigma }_{P,k}^2$$

Full text

I INTRODUCTION

Driver assistance systems that react to other vehicles e. g. adaptive cruise control (ACC) or automatic emergency braking (AEB) or collision mitigation braking (CMB) need to have a means to identify the vehicle/object that is most relevant to the system under consideration. For ACC for example the object is sought that the host vehicle is following whereas for AEB or CMB the object is sought that the host vehicle is most likely to crash into. In ACC systems this function is often referred to as track selection or target selection or target object selection (TOS).

The common approach to tackle this problem has been to obtain a fused and filtered estimate of the host vehicle path based on all available proprioceptive and exteroceptive sensor data. There is ample literature on obtaining an improved estimate of the host vehicle path by fusing and filtering inertial data (wheel speeds, yaw rate) with video line markings, road boundaries, common motion of other vehicles, and other clues about the host vehicle’s path (HVP), see e. g. [1, 2, 3]. After having obtained an HVP estimate the closest object inside this path is selected as ACC target. This approach works well if the estimated path is not very noisy and in particular does not veer over several lane widths on a time scale much shorter than that of a normal lane change. However, if the path estimate exhibits fast fluctuations which can happen e. g. if the path is only based on inertial data, the path assignment might fail occasionally. Modifying the path filter such that those fast fluctuations will be suppressed may also make the track selection less sensitive in situations where a fast object selection or deselection is required such as in lane changes of the host vehicle or the target vehicle. Instead we propose methods that postpone the filtering which is commonly applied to the host vehicle path to the final quantity, namely the lane assignment itself.

The topic of path or lane assignment has already been addressed in the following articles: In [4] the vehicle’s lane states are modeled by a three-state Hidden Markov Model to which the Viterbi algorithm to estimate the optimal lane assignment sequence as well as a so-called Lane Filter to recursively estimate the optimal current lane assignment is applied. This Lane Filter turns out to be a discrete Bayes filter followed by a MAP estimator.

In [5] the lane assignment is performed by fuzzy logic in the lateral position and lateral velocity in circular path coordinates from which an unfiltered quantity called lane-probability is derived.

In [6] the normalized lateral position (NLP) is used as a continuous generalization of the discrete lane number. This NLP is computed by propagation of probabilities using the unscented transform and is based upon the target vehicle’s position in Cartesian host vehicle coordinates and (possibly fused) lane curvature and lane width information. The discrete lane assignment is performed using postulated lane occupancy likelihood functions and contains no filtering.

In [7] two methods – a discrete and a continuous one – of a probabilistic lane assignment are described. The continuous method describes the computation of a continuous lane “number” similar to the NLP from [6]. This continuous lane number is computed at every time step by propagation of probability distributions to second order and is not filtered. The discrete method computes the probabilities of the host vehicle being in three different lanes as well as the probabilities of the target vehicle being in those lanes and then computes their joint probabilities by multiplication thereby implicitly assuming statistical independence. Again, those joint probabilities are not filtered. Both methods use an approximation of a circular arc by a third order polynomial and also approximate the lateral position of the target objects perpendicular to the estimated curved path by the lateral position with respect to a host vehicle fixed Cartesian coordinate system.

In [8] an incremental sampling of the probability distributions of target vehicles and the host vehicle path was applied to the non-linear equation for the time to cutin/cutout.

In all of the above described approaches (except [4]) filtering is not performed on the lane index or the NLP but occurs at earlier stages of the estimation. In contrast we propose here two filters where the filtering is applied to the final quantity - the path index - or the lateral position in path coordinates which is related to the path index by algebraic discretization. In the first technique presented in this paper we apply a discrete Bayes filter to the lane index - similar to the Lane Filter from [4] - albeit with adaptive measurement likelihoods that take into account the uncertainties of all measured quantities and a velocity-dependent Markov transition matrix with a band structure.

This paper is organized as follows: in the first section we illustrate the concept of a host vehicle path and introduce the two new methods to be compared as well as the comparison setup. Then we describe the two path assignment approaches, namely the Discrete Path Assignment Filter and the Continuous Path Assignment Filter and their corresponding estimators. In the fourth section we present experimental results of the comparison of the two approaches based on an unfiltered path estimate using yaw rate and host vehicle speed. A review of the probability distribution transformation from Cartesian to circular path coordinates is relegated to the appendix.

II Comparison setup

We want to assign vehicles/objects ahead of the host vehicle either to its path or its neighboring “lanes”. Note that the host vehicle path need not coincide with traffic lanes delimited by line markings; this is the case e. g. in lane change scenarios, see fig. 1. Here we also distinguish between the host vehicle (HV) coordinates $(x,y)$ which describe a Cartesian, vehicle fixed coordinate system and the host vehicle path (HVP) coordinates $(x_{P},y_{P})$ which describe a locally orthogonal coordinate system whose origin is also at the host vehicle and whose curved $x-$axis follows the HVP.

The variables $\{y^{bnd,i}_{P}\}$ with $i=1\dots 4$ delimit the host vehicle path and neighboring “lanes” along the HVP. We want to estimate the HVP index $l\in L=\{0,1,2,3,4\}$ for all detected vehicles; its interpretation is given in table I.

As an input to the HVP assignment filter we assume a list of vehicle objects - possibly based upon multi-sensor fusion - with kinematic attributes $(x,y,\dot{x},\dot{y})$ in Cartesian vehicle fixed coordinates. We also assume sensor input with cues about the vehicle path: e. g. an “inertial” path based upon yaw rate and speed (see also the appendix) as well as line markings detected by video. These individual path cues are then fed into the HVP assignment filter module depicted in dark gray in fig. 2. The first stage in the HVP assignment filter module is the transformation of the objects’ probability distribution functions (pdfs) from HV coordinates to HVP coordinates using the individual path cues, also represented by pdfs. The crucial point here is that this transformation as well as the original path cue sensor inputs should not contain any filters in order to postpone filtering to the final stage. In particular, yaw rate, speed, detected lines, etc, should be raw measurements. The objects’ pdfs in HVP coordinates are then fed into two different filters: one is a discrete Bayes filter with a discrete posterior path index distribution as output, the other is a continuous Bayes filter represented by a Kalman filter with a posterior lateral path coordinate distribution as output. This continuous distribution is discretized using the measurement likelihood function of the discrete Bayes filter. Then median estimators are applied to both distributions in order to arrive at an estimate of the path index.

In the next two sections the two filter choices - the discrete and the continuous Bayes filter - are introduced.

III Discrete Path Assignment Filter

The probability of an assigned lane $l$ at time $t_{k}$ depends on the history of the lateral positions of objects, i. e. $p(l_{k}|Z_{k})$ where $Z_{k}=\{z_{P,k},z_{P,k-1},\dots\}$. Here, $z_{P,k}$ denotes the set of measurements at time $t_{k}$ containing the lateral position of the object in path coordinates $y^{obj}_{P,k}$ and the set of measured lane boundaries in path coordinates $\{y^{bnd,1}_{P,k},y^{bnd,2}_{P,k},y^{bnd,3}_{P,k},y^{bnd,4}_{P,k}\}$:

[TABLE]

In many situations only the two innermost boundaries $y^{bnd,2}_{P,k},y^{bnd,3}_{P,k}$ will be detected by a video system and the position of the other two boundaries will then have to be extrapolated from the width of the inner lane. If no lane boundary is detected then also default values for the innermost boundaries will have to be used.

To this filtering problem a discrete Bayes filter can be applied, for a review see e. g. [9, 10].

III-A Bayes filter update

The update step of a discrete Bayes filter for this setup reads

[TABLE]

Hence we need to model the measurement likelihood $p(z_{P,k}|l_{k})$.

III-A1 Modeling of measurement likelihoods

The Bayes update requires an expression for the measurement likelihood $p(z_{P,k}|l_{k})$. However, it is not straightforward to model this likelihood since the state $l_{k}$ with five discrete values is much simpler than the set of measurements $z_{P,k}$. Therefore we will specify the inverse measurement model $p(l_{k}|z_{P,k})$ and then relate it to the measurement likelihood by using Bayes’ theorem.

By Bayes’ theorem the measurement likelihood is related to the inverse measurement model by

[TABLE]

Assuming a uniform a priori distribution for $p(l_{k})$ the quotient in eq. (3) cancels in the Bayes update expression and eq. (2) becomes

[TABLE]

To compute the probability that lane $l$ is occupied at $t_{k}$ we compute the probability that the object is left of the right boundary $y^{bnd,l+1}_{P,k}$ and subtract the probability that the object is left of the left lane boundary $y^{bnd,l}_{P,k}$, i. e.

[TABLE]

If the individual probability distributions for the lateral positions in HVP coordinates of the object and the lane boundaries are represented by ${\mathcal{N}}(y^{obj}_{P,k};\mu^{obj}_{P,k},\sigma^{obj}_{P,k})$ and ${\mathcal{N}}(y^{bnd,l}_{P,k};\mu^{bnd,l}_{P,k},\sigma^{bnd,l}_{P,k})$ the integrands in eq. (5) above are given by

[TABLE]

where $\tilde{y}^{bnd,l}_{P,k}:=y^{obj}_{P,k}-y^{bnd,l}_{P,k}$. Then

[TABLE]

where $\Phi$ is the cumulative distribution function of the standard normal distribution. By using the inverse measurement model we have derived the Bayes update step under the mild assumption of a uniformly distributed a priori lane assignment probability. This approach differs from the one in [6] where the measurement model $p(z_{P,k}|l_{k})$ was postulated in a piecewise, heuristic manner.

The parameters $\mu^{bnd,l}_{P,k},\sigma^{bnd,l}_{P,k}$ constitute input from the transformation module (“Transform objects to path coordinates”) in fig. 2. As an illustration the inverse measurement likelihoods have been plotted in fig. 3 and fig. 4 for two different values of the effective variance $(\sigma^{l}_{eff,k})^{2}=(\sigma^{obj}_{P,k})^{2}+(\sigma^{bnd,l}_{P,k})^{2}$ where the variances of all lane boundaries were taken to be identical.

It can be seen that for a small effective variance the lane assignment likelihoods stay roughly constant over almost the entire width of the lane whereas for large effective variances the likelihoods approximate a Gaussian shape except for the boundary likelihoods. Integration of the likelihood functions over $\mu^{obj}_{P,k}$ yields the width of the lane which is infinite for the likelihoods of the boundary states ’left of left path’ and ’right of right path’.

III-B Bayes filter prediction

The Bayes filter prediction is given by

[TABLE]

where the Markov transition matrix $p(l_{k}|l_{k-1})$ is modeled as a perturbation of the identity matrix, i. e. objects tend to stay in their lanes, with small probabilities for transitions to neighboring lanes, but not to next-to-neighboring lanes.111A similar transition structure albeit with partly different states was used in [11].

[TABLE]

Here, $\epsilon$ denotes the default probability for transitions between neighboring lanes and $\eta$ is an additional increment or decrement parameter, respectively, that can depend for example upon the lateral velocity of the object or upon a detected indicator signal. The specific expressions involving $\eta$ arise from the following assumptions: the velocity perturbation is additive; a velocity to the right should increase the transition probability from the current lane the right lane and decrease the probability to the left path; and the columns of the Markov transition matrix must add up to one for proper normalization.

III-C Estimator for HVP assignment

After having obtained an a posteriori probability distribution of lane indices by discrete Bayes filtering a single lane index estimate needs to be extracted by an estimator. This is done by a median estimator, i. e. by determining the lane index $l_{k}^{est}$ for which

[TABLE]

If the probability of this lane estimate is also above a certain threshold $p(l_{k}^{est}|Z^{k})\geq p^{l}_{min}$ then this estimate is accepted as an assigned lane for use by a subsequent track selection; otherwise no lane will be assigned to this object. We have decided against a mean estimator because it is less robust and against a MAP estimator because for strongly oscillating host vehicle paths the lane probability distribution can exhibit several maxima of similar height. An example distribution where the left lane is assigned is depicted in fig. 5.

IV Continuous Path Assignment Filter

The first step in the continuous case is the filtering of the object’s lateral position in HVP coordinates: $y^{obj}_{P}$. Then, its continuous distribution function is mapped onto a discrete lane assignment distribution using path boundary information. Finally, a median estimator is applied to extract a single lane assignment value.

IV-A Dynamical system for lateral HVP coordinate

The state is $\hat{y}^{obj}_{P}$ where the diacritic $\hat{}$ serves to distinguish this filtered quantity from the unfiltered measurement $y^{obj}_{P}$ in eq. (1). In order to be able to compare results using continuous filtering to discrete filtering on an equal footing the velocity is modeled as an input $u_{k}$ to the dynamical system in analogy to (8) and is not part of the state. However, inclusion of derivatives (velocity, acceleration, …) of the lateral position in the state is straightforward and appropriate for more dynamic situations. The dynamical model is a discrete-time counterpart white noise velocity model with

[TABLE]

with $\Delta t_{k}=t_{k+1}-t_{k}$ and $\nu_{k}$ the process noise. The measurement equation is

[TABLE]

where $z_{k}$ is the lateral position in host vehicle path coordinates. For circular paths arising from host vehicle yaw rate and velocity, the lateral position is given by eq. (13) as described in the appendix. For non-circular paths, this lateral position can be provided by other path cues as indicated in fig. 2. To this one-dimensional system a Kalman filter is applied yielding a posterior distribution ${\mathcal{N}}(\hat{y}^{obj}_{P,k};\hat{\mu}^{obj}_{P,k},\hat{\sigma}^{obj}_{P,k})$. In order to satisfy one of the prerequisites for a Kalman filter we have also verified in the appendix that the propagation of pdfs of the parameters of eq. (13) by Taylor expansion is a valid approximation; hence $z_{k}$ is given by eq. (VII) from the appendix and the variance of the measurement noise $w_{k}$ is given by eq. (15).

IV-B Map to discrete lane distribution and estimator

With the lane boundary information contained in $z_{P,k}$ (from eq. (1)) the inverse measurement likelihood eq. (6) can be applied to arrive at a discrete probability distribution for the five path assignment possibilities from table I. Note that in this case the mean and standard deviation $\mu^{obj}_{P,k},\sigma^{obj}_{P,k}$ are replaced by $\hat{\mu}^{obj}_{P,k},\hat{\sigma}^{obj}_{P,k}$ from the filtered pdf of $\hat{y}^{obj}_{P,k}$. Finally, as in sec. III-C a median estimator is applied followed by thresholding.

V Experimental results

We have evaluated both algorithms using recorded experimental data from highway and country road scenarios with a total length of ca. 117 minutes of ground truthed data. Here path cue inputs from the camera were disabled, hence the only source of path information was due to the inertial host vehicle signals yaw rate and speed. For a representative comparison of both approaches we have reduced the number of free parameters in each algorithm to one: namely the value of the default transition to neighboring lanes $\epsilon$ for the Discrete Path Assignment Filter and the standard deviation $\sigma_{\nu}$ of the process noise for the Continuous Path Assignment Filter. Other parameters such as sensor characteristics or the thresholds for the median estimators are identical. With those parameters in the ranges $\epsilon\in\{1e^{-1}...1e^{-6}\}$ and $\sigma_{\nu}\in\{0.04...0.4m/s\}$ the ROC curve in fig. 6 was generated.

It can be seen that the Continuous Path Assignment Filter has a better true positive (TP) rate than the Discrete Path Assignment Filter whereas the false positive (FP) rate can be lower for the Discrete Path Assignment Filter albeit at the cost of a low TP rate. For comparison with a path assignment method where filtering is done at the path estimation level and not at the lane assignment level we have also plotted the ROC point of an assignment method where the path itself is filtered and estimated and the path assignment is done by projecting the vehicles’ position into this path and the neighboring paths and evaluating the assigned path geometrically using its current lateral position (“Path Filter with geom. Assignment”).

The overall worse performance of the discrete filter might be due the fact that the discrete filter contains less information than the continuous one because the discrete probability distribution consists of five values for each lane or path. Hence information whether the target vehicle is directly in the middle of a lane or displaced to the left or right boundary is averaged over by a single probability for this lane. In contrast, the continuous filter generates a continuous a posteriori distribution for the lateral displacement with respect to the host vehicle trajectory $y^{obj}_{P,k}$. The subsequent discretization is just an algebraic map without additional filtering.

VI Conclusions

We have comparatively evaluated two techniques of path assignment that were motivated by the requirement to postpone filtering as late as possible in the estimation process. In the discrete case the filtering was indeed performed on the final discrete path index distribution, in the continuous case the filtering was performed on the lateral displacement followed by discretization. We have derived path index measurement likelihoods - used by both techniques - from first principles making the minimal assumption of a uniform a priori distribution.

Our numerical study suggests that the continuous filtering although the filtering is not directly done on the final quantity has advantages with respect to the discrete approach. Furthermore, in addition to the numerical evidence in favor of the continuous filtering, the discrete approach contains more modeling assumptions: the assignment probability is averaged over the width of a lane in the filtering stage - as discussed above, and the prediction requires modeling assumptions about lane transitions which were subsumed in this paper into the parameters $\epsilon$ and $\eta$. We have also shown that in relevant parameter regions of forward looking sensors the propagation of probability distributions by Taylor expansion of formula (13) is a valid approximation.

VII Appendix: Inertial HVP transformation

If no lane markings are detected and no other information about the HVP such as guard rails are available, then the only source of information about the HVP are inertial host vehicle data, i. e. the host vehicle speed $v$ and its yaw rate $\dot{\psi}$. Using the common assumption of a steady-state circular motion the vehicle will travel on a circular arc with radius $r={v\over\dot{\psi}}$ as depicted in fig. 7.

Here we also allow for a non-vanishing heading angle $\alpha$, i. e. the host vehicle path at the host vehicle’s reference point – the middle of the front bumper – is not tangential to the vehicle’s longitudinal axis but is rotated by an angle $\alpha$

[TABLE]

Then the lateral distance $y_{P}$ in HVP coordinates $(x_{P},y_{P})$ of an object with HV coordinates $(x,y)$ is given by (see also [5, 6])

[TABLE]

This is the minimal distance to the estimated HVP. In a probabilistic framework the parameters $\mu_{P,k},\sigma_{P,k}$ of the pdf of $y_{P,k}$ must be determined. The pdfs of the signals in eq. (13) are propagated by Taylor expansion, i. e. given the vector $\zeta_{k}=({v}_{k},{\dot{\psi}}_{k},{x}_{k},{y}_{k})^{\top}$, its mean $\mu_{\zeta_{k}}=(\mu^{v}_{k},\mu^{\dot{\psi}}_{k},\mu^{x}_{k},\mu^{y}_{k})^{\top}$ and its covariance matrix $V_{\zeta_{k}}$ we get

[TABLE]

Here, $\partial_{\zeta}y_{P,k}\left(\mu_{\zeta_{k}}\right)$ denotes the Jacobi matrix of $y_{P,k}$ at $\mu_{\zeta_{k}}$. In order to validate that Taylor expansion results in an accurate approximation of the pdf of eq. (13) we have performed a Monte-Carlo study where 24576 different pdfs of $x,y,v,$ and $\dot{\psi}$ were propagated through (13) by sampling with 5000 samples. The values for $x$ ranged between one and $110m$, the value for $y$ was calculated by letting the angle range between $-21$ and $21$ degrees. Velocities of $1$ to $70m/s$ were taken into account and the yaw rates considered ranged from $-0.7$ to $0.7rad/s$. Then we compared the resulting normalized histograms to Gaussian distributions parametrized by eqs. (VII) and (15). For comparison of two pdfs the Hellinger distance was chosen since it satisfies all conditions of a mathematical metric. In fig. 8 the distribution of Hellinger distances between the Monte-Carlo run and the Gaussian from Taylor expansion is plotted for all 24576 pdfs.

The largest distance value is ca. $3$. In order to assess the consequence of such a value we have plotted the normalized histogram with the largest Hellinger distance together with its corresponding Gaussian in fig. 9. It can be seen that the histogram with that distance still approximates the Gaussian to high accuracy. Hence we conclude that in the chosen parameter ranges the propagation of pdfs by Taylor expansion is appropriate.