CZU-MHAD: A multimodal dataset for human action recognition utilizing a depth camera and 10 wearable inertial sensors

TL;DR

This paper introduces CZU-MHAD, a comprehensive multi-modal human action dataset combining depth videos, skeleton data, and inertial signals from wearable sensors, enabling more accurate action recognition research.

Contribution

The paper presents a new, publicly available multi-modal dataset with synchronized depth, skeleton, and inertial data for 22 actions, addressing limitations of previous datasets.

Findings

Dataset effectively captures complex human actions.

Multi-modal data improves action recognition accuracy.

Experimental results validate dataset usefulness for structural and fusion studies.

Abstract

Human action recognition has been widely used in many fields of life, and many human action datasets have been published at the same time. However, most of the multi-modal databases have some shortcomings in the layout and number of sensors, which cannot fully represent the action features. Regarding the problems, this paper proposes a freely available dataset, named CZU-MHAD (Changzhou University: a comprehensive multi-modal human action dataset). It consists of 22 actions and three modals temporal synchronized data. These modals include depth videos and skeleton positions from a kinect v2 camera, and inertial signals from 10 wearable sensors. Compared with single modal sensors, multi-modal sensors can collect different modal data, so the use of multi-modal sensors can describe actions more accurately. Moreover, CZU-MHAD obtains the 3-axis acceleration and 3-axis angular velocity of 10 main motion joints by binding inertial sensors to them, and these data were captured at the same time. Experimental results are provided to show that this dataset can be used to study structural relationships between different parts of the human body when performing actions and fusion approaches that involve multi-modal sensor data.