Gesture Matters: Pedestrian Gesture Recognition for AVs Through Skeleton Pose Evaluation
Alif Rizqullah Mahdi, Mahdi Rezaei, Natasha Merat

TL;DR
This paper introduces a gesture recognition framework for autonomous vehicles using 2D pose estimation, achieving 87% accuracy in classifying pedestrian gestures to enhance traffic interaction understanding.
Contribution
The study presents a novel gesture classification method based on 2D skeleton pose evaluation, focusing on real-world data and key features like hand position and velocity.
Findings
Achieved 87% classification accuracy.
Hand position and velocity are highly discriminative.
Improves AV perception of pedestrian gestures.
Abstract
Gestures are a key component of non-verbal communication in traffic, often helping pedestrian-to-driver interactions when formal traffic rules may be insufficient. This problem becomes more apparent when autonomous vehicles (AVs) struggle to interpret such gestures. In this study, we present a gesture classification framework using 2D pose estimation applied to real-world video sequences from the WIVW dataset. We categorise gestures into four primary classes (Stop, Go, Thank & Greet, and No Gesture) and extract 76 static and dynamic features from normalised keypoints. Our analysis demonstrates that hand position and movement velocity are especially discriminative in distinguishing between gesture classes, achieving a classification accuracy score of 87%. These findings not only improve the perceptual capabilities of AV systems but also contribute to the broader understanding of…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsHuman Pose and Action Recognition · Hand Gesture Recognition Systems · Social Robot Interaction and HRI
