Machine learning models and facial regions videos for estimating heart rate: a review on Patents, Datasets and Literature

TL;DR

This review explores machine learning methods for estimating heart rate from facial videos, analyzing patents, datasets, and literature to identify techniques, models, and challenges in non-invasive cardiac monitoring.

Contribution

It provides a comprehensive overview of existing patents, datasets, and machine learning techniques used for heart rate estimation from facial videos, highlighting current advancements and challenges.

Findings

Several machine learning models like EVM-CNN and VGG-16 are used for heart rate estimation.

Most datasets are for academic research with varied annotations.

Techniques include region of interest extraction and Video Magnification.

Abstract

Estimating heart rate is important for monitoring users in various situations. Estimates based on facial videos are increasingly being researched because it makes it possible to monitor cardiac information in a non-invasive way and because the devices are simpler, requiring only cameras that capture the user's face. From these videos of the user's face, machine learning is able to estimate heart rate. This study investigates the benefits and challenges of using machine learning models to estimate heart rate from facial videos, through patents, datasets, and articles review. We searched Derwent Innovation, IEEE Xplore, Scopus, and Web of Science knowledge bases and identified 7 patent filings, 11 datasets, and 20 articles on heart rate, photoplethysmography, or electrocardiogram data. In terms of patents, we note the advantages of inventions related to heart rate estimation, as described by the authors. In terms of datasets, we discovered that most of them are for academic purposes and with different signs and annotations that allow coverage for subjects other than heartbeat estimation. In terms of articles, we discovered techniques, such as extracting regions of interest for heart rate reading and using Video Magnification for small motion extraction, and models such as EVM-CNN and VGG-16, that extract the observed individual's heart rate, the best regions of interest for signal extraction and ways to process them.