Gesture Recognition with a Skeleton-Based Keyframe Selection Module

TL;DR

This paper introduces a novel bidirectional two-pathway network for gesture recognition that effectively combines skeleton-based keyframe selection with temporal attention, improving spatial and temporal feature extraction.

Contribution

It presents a new BCCN architecture with a keyframe selection module and dual pathways, enhancing gesture recognition accuracy over existing methods.

Findings

Improved gesture recognition performance on multiple datasets

Better activation maps for spatial and temporal features

Effective integration of keyframe selection with temporal attention

Abstract

We propose a bidirectional consecutively connected two-pathway network (BCCN) for efficient gesture recognition. The BCCN consists of two pathways: (i) a keyframe pathway and (ii) a temporal-attention pathway. The keyframe pathway is configured using the skeleton-based keyframe selection module. Keyframes pass through the pathway to extract the spatial feature of itself, and the temporal-attention pathway extracts temporal semantics. Our model improved gesture recognition performance in videos and obtained better activation maps for spatial and temporal properties. Tests were performed on the Chalearn dataset, the ETRI-Activity 3D dataset, and the Toyota Smart Home dataset.