Gesture Matters: Pedestrian Gesture Recognition for AVs Through Skeleton Pose Evaluation

TL;DR

This paper introduces a gesture recognition framework for autonomous vehicles using 2D pose estimation, achieving 87% accuracy in classifying pedestrian gestures to enhance traffic interaction understanding.

Contribution

The study presents a novel gesture classification method based on 2D skeleton pose evaluation, focusing on real-world data and key features like hand position and velocity.

Findings

Achieved 87% classification accuracy.

Hand position and velocity are highly discriminative.

Improves AV perception of pedestrian gestures.

Abstract

Gestures are a key component of non-verbal communication in traffic, often helping pedestrian-to-driver interactions when formal traffic rules may be insufficient. This problem becomes more apparent when autonomous vehicles (AVs) struggle to interpret such gestures. In this study, we present a gesture classification framework using 2D pose estimation applied to real-world video sequences from the WIVW dataset. We categorise gestures into four primary classes (Stop, Go, Thank & Greet, and No Gesture) and extract 76 static and dynamic features from normalised keypoints. Our analysis demonstrates that hand position and movement velocity are especially discriminative in distinguishing between gesture classes, achieving a classification accuracy score of 87%. These findings not only improve the perceptual capabilities of AV systems but also contribute to the broader understanding of pedestrian behaviour in traffic contexts.