Hand gesture detection in tests performed by older adults

TL;DR

This paper presents a new hand gesture detection system for remote testing of older adults, achieving higher accuracy than existing methods and demonstrating robustness to noisy data, facilitating large-scale remote health assessments.

Contribution

The study compares various network architectures for hand gesture detection, improves accuracy with attention layers, and evaluates performance on noisy datasets with blurred images.

Findings

Detection mAP of 0.782 surpasses state-of-the-art.

Attention layers improve classification accuracy for similar gestures.

Network performs better on noisy, blurred image datasets.

Abstract

Our team are developing a new online test that analyses hand movement features associated with ageing that can be completed remotely from the research centre. To obtain hand movement features, participants will be asked to perform a variety of hand gestures using their own computer cameras. However, it is challenging to collect high quality hand movement video data, especially for older participants, many of whom have no IT background. During the data collection process, one of the key steps is to detect whether the participants are following the test instructions correctly and also to detect similar gestures from different devices. Furthermore, we need this process to be automated and accurate as we expect many thousands of participants to complete the test. We have implemented a hand gesture detector to detect the gestures in the hand movement tests and our detection mAP is 0.782 which is better than the state-of-the-art. In this research, we have processed 20,000 images collected from hand movement tests and labelled 6,450 images to detect different hand gestures in the hand movement tests. This paper has the following three contributions. Firstly, we compared and analysed the performance of different network structures for hand gesture detection. Secondly, we have made many attempts to improve the accuracy of the model and have succeeded in improving the classification accuracy for similar gestures by implementing attention layers. Thirdly, we have created two datasets and included 20 percent of blurred images in the dataset to investigate how different network structures were impacted by noisy data, our experiments have also shown our network has better performance on the noisy dataset.