Holoscopic 3D Micro-Gesture Database for Wearable Device Interaction

TL;DR

This paper introduces a high-quality holoscopic 3D micro-gesture database to advance HCI research for AR and VR, enabling improved micro-gesture recognition with new data resources.

Contribution

The creation of the first holoscopic 3D micro-gesture database using a specialized camera, including a large dataset of images and videos for gesture recognition research.

Findings

Initial recognition results using traditional features and classifiers are promising.

The database will facilitate further research and development in micro-gesture HCI.

High-quality 3D gesture data enhances recognition accuracy and robustness.

Abstract

With the rapid development of augmented reality (AR) and virtual reality (VR) technology, human-computer interaction (HCI) has been greatly improved for gaming interaction of AR and VR control. The finger micro-gesture is one of the important interactive methods for HCI applications such as in the Google Soli and Microsoft Kinect projects. However, the progress in this research is slow due to the lack of high quality public available database. In this paper, holoscopic 3D camera is used to capture high quality micro-gesture images and a new unique holoscopic 3D micro-gesture (HoMG) database is produced. The principle of the holoscopic 3D camera is based on the fly viewing system to see the objects. HoMG database recorded the image sequence of 3 conventional gestures from 40 participants under different settings and conditions. For the purpose of micro-gesture recognition, HoMG has a video subset with 960 videos and a still image subset with 30635 images. Initial micro-gesture recognition on both subsets has been conducted using traditional 2D image and video features and popular classifiers and some encouraging performance has been achieved. The database will be available for the research communities and speed up the research in this area.