A Prototype-Based Generalized Zero-Shot Learning Framework for Hand Gesture Recognition

TL;DR

This paper introduces a prototype-based generalized zero-shot learning framework for hand gesture recognition, enabling the recognition of both seen and unseen gestures by leveraging semantic representations and a dual-branch architecture.

Contribution

It presents an end-to-end prototype-based GZSL framework with a new hand gesture dataset, advancing recognition capabilities for unseen gesture categories.

Findings

Effective recognition of unseen gestures demonstrated

Proposed framework outperforms baseline methods

New hand gesture dataset tailored for GZSL

Abstract

Hand gesture recognition plays a significant role in human-computer interaction for understanding various human gestures and their intent. However, most prior works can only recognize gestures of limited labeled classes and fail to adapt to new categories. The task of Generalized Zero-Shot Learning (GZSL) for hand gesture recognition aims to address the above issue by leveraging semantic representations and detecting both seen and unseen class samples. In this paper, we propose an end-to-end prototype-based GZSL framework for hand gesture recognition which consists of two branches. The first branch is a prototype-based detector that learns gesture representations and determines whether an input sample belongs to a seen or unseen category. The second branch is a zero-shot label predictor which takes the features of unseen classes as input and outputs predictions through a learned mapping mechanism between the feature and the semantic space. We further establish a hand gesture dataset that specifically targets this GZSL task, and comprehensive experiments on this dataset demonstrate the effectiveness of our proposed approach on recognizing both seen and unseen gestures.