MHAD: Multimodal Home Activity Dataset with Multi-Angle Videos and Synchronized Physiological Signals

TL;DR

The MHAD dataset provides a comprehensive collection of multi-angle videos and physiological signals in a real home environment, facilitating research in non-contact health monitoring.

Contribution

This paper introduces the MHAD dataset, the first to combine multi-angle videos with five physiological signals for passive home monitoring research.

Findings

Validated with multiple rPPG methods

Supports unsupervised and supervised physiological signal extraction

Enhances real-world home health monitoring research

Abstract

Video-based physiology, exemplified by remote photoplethysmography (rPPG), extracts physiological signals such as pulse and respiration by analyzing subtle changes in video recordings. This non-contact, real-time monitoring method holds great potential for home settings. Despite the valuable contributions of public benchmark datasets to this technology, there is currently no dataset specifically designed for passive home monitoring. Existing datasets are often limited to close-up, static, frontal recordings and typically include only 1-2 physiological signals. To advance video-based physiology in real home settings, we introduce the MHAD dataset. It comprises 1,440 videos from 40 subjects, capturing 6 typical activities from 3 angles in a real home environment. Additionally, 5 physiological signals were recorded, making it a comprehensive video-based physiology dataset. MHAD is compatible with the rPPG-toolbox and has been validated using several unsupervised and supervised methods. Our dataset is publicly available at https://github.com/jdh-algo/MHAD-Dataset.