All Weather Perception: Joint Data Association, Tracking, and Classification for Autonomous Ground Vehicles

TL;DR

This paper introduces a real-time probabilistic perception algorithm that combines data association, tracking, and classification for autonomous ground vehicles, effectively functioning in all-weather conditions by integrating multiple sensor modalities.

Contribution

It extends a Rao-Blackwellized Particle Filter to include multiple model tracking for classification and demonstrates its effectiveness with sensor fusion in adverse weather.

Findings

Robust data association, tracking, and classification in all-weather conditions.

Effective sensor fusion of camera, lidar, and radar sensors.

Experimental validation with upgraded Cornell AGV in challenging environments.

Abstract

A novel probabilistic perception algorithm is presented as a real-time joint solution to data association, object tracking, and object classification for an autonomous ground vehicle in all-weather conditions. The presented algorithm extends a Rao-Blackwellized Particle Filter originally built with a particle filter for data association and a Kalman filter for multi-object tracking (Miller et al. 2011a) to now also include multiple model tracking for classification. Additionally a state-of-the-art vision detection algorithm that includes heading information for autonomous ground vehicle (AGV) applications was implemented. Cornell's AGV from the DARPA Urban Challenge was upgraded and used to experimentally examine if and how state-of-the-art vision algorithms can complement or replace lidar and radar sensors. Sensor and algorithm performance in adverse weather and lighting conditions is tested. Experimental evaluation demonstrates robust all-weather data association, tracking, and classification where camera, lidar, and radar sensors complement each other inside the joint probabilistic perception algorithm.