Fine-grained Hand Gesture Recognition in Multi-viewpoint Hand Hygiene

TL;DR

This paper introduces a new dataset called MFH for fine-grained hand gesture recognition in multi-viewpoint hand hygiene, addressing data mismatch and viewpoint variation issues, and demonstrates the effectiveness of self-supervised learning methods on this dataset.

Contribution

The paper presents the MFH dataset with over 730,000 samples for hand hygiene gesture recognition and proposes using self-supervised learning to improve recognition performance.

Findings

Self-supervised learning achieves competitive accuracy.

MFH dataset covers diverse viewpoints and hygiene steps.

Method outperforms baseline models on MFH.

Abstract

This paper contributes a new high-quality dataset for hand gesture recognition in hand hygiene systems, named "MFH". Generally, current datasets are not focused on: (i) fine-grained actions; and (ii) data mismatch between different viewpoints, which are available under realistic settings. To address the aforementioned issues, the MFH dataset is proposed to contain a total of 731147 samples obtained by different camera views in 6 non-overlapping locations. Additionally, each sample belongs to one of seven steps introduced by the World Health Organization (WHO). As a minor contribution, inspired by advances in fine-grained image recognition and distribution adaptation, this paper recommends using the self-supervised learning method to handle these preceding problems. The extensive experiments on the benchmarking MFH dataset show that the introduced method yields competitive performance in both the Accuracy and the Macro F1-score. The code and the MFH dataset are available at https://github.com/willogy-team/hand-gesture-recognition-smc2021.