VIPL-HR: A Multi-modal Database for Pulse Estimation from Less-constrained Face Video

TL;DR

This paper introduces VIPL-HR, a large-scale multi-modal database for remote pulse estimation from face videos in less-constrained scenarios, and proposes a deep learning method called RhythmNet that achieves promising results.

Contribution

The paper provides the first large-scale, multi-modal database for remote HR estimation in unconstrained conditions and develops a novel deep learning approach for accurate pulse estimation.

Findings

RhythmNet achieves high accuracy on VIPL-HR and public datasets.

VIPL-HR includes diverse variations like head movements and illumination.

Deep learning improves remote HR estimation in less-controlled environments.

Abstract

Heart rate (HR) is an important physiological signal that reflects the physical and emotional activities of humans. Traditional HR measurements are mainly based on contact monitors, which are inconvenient and may cause discomfort for the subjects. Recently, methods have been proposed for remote HR estimation from face videos. However, most of the existing methods focus on well-controlled scenarios, their generalization ability into less-constrained scenarios are not known. At the same time, lacking large-scale databases has limited the use of deep representation learning methods in remote HR estimation. In this paper, we introduce a large-scale multi-modal HR database (named as VIPL-HR), which contains 2,378 visible light videos (VIS) and 752 near-infrared (NIR) videos of 107 subjects. Our VIPL-HR database also contains various variations such as head movements, illumination variations, and acquisition device changes. We also learn a deep HR estimator (named as RhythmNet) with the proposed spatial-temporal representation, which achieves promising results on both the public-domain and our VIPL-HR HR estimation databases. We would like to put the VIPL-HR database into the public domain.