Construct Dynamic Graphs for Hand Gesture Recognition via Spatial-Temporal Attention
Yuxiao Chen, Long Zhao, Xi Peng, Jianbo Yuan, and Dimitris N. Metaxas

TL;DR
This paper introduces DG-STA, a dynamic graph-based method utilizing spatial-temporal attention for hand gesture recognition, achieving high accuracy and efficiency by automatically learning graph features and reducing computational costs.
Contribution
The paper presents a novel dynamic graph construction with self-attention for hand gesture recognition, significantly reducing computational cost and improving robustness in challenging conditions.
Findings
Outperforms state-of-the-art on DHG-14/28 and SHREC'17 datasets.
Reduces computational cost by 99% with a new spatial-temporal mask.
Demonstrates robustness in challenging recognition scenarios.
Abstract
We propose a Dynamic Graph-Based Spatial-Temporal Attention (DG-STA) method for hand gesture recognition. The key idea is to first construct a fully-connected graph from a hand skeleton, where the node features and edges are then automatically learned via a self-attention mechanism that performs in both spatial and temporal domains. We further propose to leverage the spatial-temporal cues of joint positions to guarantee robust recognition in challenging conditions. In addition, a novel spatial-temporal mask is applied to significantly cut down the computational cost by 99%. We carry out extensive experiments on benchmarks (DHG-14/28 and SHREC'17) and prove the superior performance of our method compared with the state-of-the-art methods. The source code can be found at https://github.com/yuxiaochen1103/DG-STA.
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsHand Gesture Recognition Systems · Human Pose and Action Recognition · Multimodal Machine Learning Applications
