A Framework for 3D Tracking of Frontal Dynamic Objects in Autonomous Cars

TL;DR

This paper presents a comprehensive framework combining deep learning, nonlinear filtering, and switching estimation techniques for real-time 3D tracking of frontal dynamic objects in autonomous vehicles, addressing depth estimation challenges with monocular cameras.

Contribution

It introduces a novel switching SDRE filter with stability analysis and real-time implementation for improved 3D object tracking in autonomous driving.

Findings

Enhanced robustness of the SDRE filter through switching estimation error covariance.

Successful real-time 3D tracking validated on Jetson TX2 with radar data.

Simulation results confirm the effectiveness of the proposed approach.

Abstract

Both recognition and 3D tracking of frontal dynamic objects are crucial problems in an autonomous vehicle, while depth estimation as an essential issue becomes a challenging problem using a monocular camera. Since both camera and objects are moving, the issue can be formed as a structure from motion (SFM) problem. In this paper, to elicit features from an image, the YOLOv3 approach is utilized beside an OpenCV tracker. Subsequently, to obtain the lateral and longitudinal distances, a nonlinear SFM model is considered alongside a state-dependent Riccati equation (SDRE) filter and a newly developed observation model. Additionally, a switching method in the form of switching estimation error covariance is proposed to enhance the robust performance of the SDRE filter. The stability analysis of the presented filter is conducted on a class of discrete nonlinear systems. Furthermore, the ultimate bound of estimation error caused by model uncertainties is analytically obtained to investigate the switching significance. Simulations are reported to validate the performance of the switched SDRE filter. Finally, real-time experiments are performed through a multi-thread framework implemented on a Jetson TX2 board, while radar data is used for the evaluation.