Traffic Control Gesture Recognition for Autonomous Vehicles

TL;DR

This paper introduces a new 3D skeleton-based dataset and deep learning models for traffic control gesture recognition to enhance autonomous vehicle understanding of traffic officer signals.

Contribution

It provides a novel dataset and evaluates multiple deep neural network architectures for traffic gesture classification in autonomous driving.

Findings

Deep neural networks achieve high accuracy on the new dataset.

Attention mechanisms improve gesture recognition performance.

The dataset and models are publicly available for further research.

Abstract

A car driver knows how to react on the gestures of the traffic officers. Clearly, this is not the case for the autonomous vehicle, unless it has road traffic control gesture recognition functionalities. In this work, we address the limitation of the existing autonomous driving datasets to provide learning data for traffic control gesture recognition. We introduce a dataset that is based on 3D body skeleton input to perform traffic control gesture classification on every time step. Our dataset consists of 250 sequences from several actors, ranging from 16 to 90 seconds per sequence. To evaluate our dataset, we propose eight sequential processing models based on deep neural networks such as recurrent networks, attention mechanism, temporal convolutional networks and graph convolutional networks. We present an extensive evaluation and analysis of all approaches for our dataset, as well as real-world quantitative evaluation. The code and dataset is publicly available.