Improving Heart Rate Variability Measurements from Consumer Smartwatches with Machine Learning

TL;DR

This paper explores how machine learning, especially neural networks, can reduce measurement errors in heart rate variability data collected from consumer smartwatches by accounting for wearer movement using additional sensor data.

Contribution

It introduces a method to minimize HRV measurement errors from smartwatches by incorporating accelerometer data through neural learning models, addressing movement-related inaccuracies.

Findings

Significant correlation between movement and HRV measurement error.

Using accelerometer data reduces measurement error.

Neural learning can effectively improve smartwatch HRV accuracy.

Abstract

The reactions of the human body to physical exercise, psychophysiological stress and heart diseases are reflected in heart rate variability (HRV). Thus, continuous monitoring of HRV can contribute to determining and predicting issues in well-being and mental health. HRV can be measured in everyday life by consumer wearable devices such as smartwatches which are easily accessible and affordable. However, they are arguably accurate due to the stability of the sensor. We hypothesize a systematic error which is related to the wearer movement. Our evidence builds upon explanatory and predictive modeling: we find a statistically significant correlation between error in HRV measurements and the wearer movement. We show that this error can be minimized by bringing into context additional available sensor information, such as accelerometer data. This work demonstrates our research-in-progress on how neural learning can minimize the error of such smartwatch HRV measurements.

Figures

Equations

$$RMSSD=\sqrt{\frac{1}{N-1}\sum _{i=1}^N(IB{I}_i-IB{I}_{i-1}{)}^2},$$

$${\epsilon }_t=RMSS{D}_{{\text{lx@glossaries@gls@linkacronymFBGleavevmodeheart rate monitor}}_t}-RMSS{D}_{{\text{lx@glossaries@gls@linkacronymGVAleavevmodeconsumer smartwatch}}_t}.$$

Full text

neural networks, heart rate variability, smartwatch

††copyright: rightsretained††journalyear: 2019††conference: Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and the 2019 International Symposium on Wearable Computers; September 9–13, 2019; London, United Kingdom††booktitle: Adjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and the 2019 International Symposium on Wearable Computers (UbiComp/ISWC ’19 Adjunct), September 9–13, 2019, London, United Kingdom††doi: 10.1145/3341162.3346276††isbn: 978-1-4503-6869-8/19/09††ccs: Human-centered computing Ubiquitous computing††ccs: Applied computing Consumer health††ccs: Computing methodologies Neural networks

1. Introduction

The ability of human heart to adapt to sudden physiological changes is visible in heart rate variability (HRV); the variation of subsequent inter-beat intervals across time. In fact, it informs about involuntary physiological functions, as it is a valid measure of the interaction between the autonomic nervous systems (i.e. sympathetic and parasympathetic). HRV reflects individual reactions to physical exercise, psychophysiological stress and heart diseases. In particular, HRV has been associated with parasympathetic activity (Goldberger et al., 2001; Dimitriev and Saperova, 2015) and hypoglycemic episodes (Cichosz et al., 2014; Schächinger et al., 2004) of type 1 diabetes mellitus patients, which are caused by a deficit in the sympathetic nervous system (Cryer, 2004).

Self-tracking devices are becoming increasingly popular: sales of sports and fitness trackers increased from 97.6 million in 2015 to 134 million in 2018 and are expected to reach 148.5 million in 2021 (Intelligence, 2018) (cf. also (Henriksen et al., 2018) for a review on availability of wrist-worn fitness wearable devices and sensors). Due to their discrete form-factor, smartwatches are suitable for continuous monitoring of the wearers vital factors. To this end, they are often equipped with measurement capabilities such as an optical heart rate sensor. Besides observing the heart rate, raw data from these sensors can as well be used to compute HRV. Their affordability and unobtrusiveness makes smartwatches relevant to a large audience and, in particular, more likely to be adapted than professional electrocardiographies (ECGs) for measuring HRV.

Wearables measuring HRV can considerably contribute to improving well-being and mental health. For instance, it can predict acute complications of type 1 diabetes mellitus (i.e., hypoglycemia). As such, its benefit to better health becomes relevant to around 40 million people worldwide and this number is expected to double within the next 10 to 20 years (Federation, 2017).

**Objective: ** This work demonstrates research-in-progress on the potential of machine learning approaches improving smartwatch HRV measurements. More specifically, we aim to utilize explanatory and predictive modeling to the sensor data provided by smartwatches with the goal of minimizing errors in HRV measurements due to physical activity.

2. Related work

2.1. Consumer Wearable Devices

Wearable devices allow non-invasive monitoring of physiological activity. In particular, smartwatches can be applied to different contexts, such as stress (Dimitriev and Saperova, 2015), sleep quality (Phan et al., 2015a) or physical fitness (Dooley et al., 2017) to measure well-being through heart rate. Moreover, these devices are often equipped with a range of additional sensors, allowing for more holistic measurements. For instance, there are wrist-based consumer smartwatches that provide multiple data dimensions such as inter-beat intervals (obtained via optical sensor), three-axis accelerometer data, steps, burned calories or proprietary stress values. However, such non-professional devices need to be compared to a more precise instrument. The Firstbeat Bodyguard 2 (heart rate monitor) is an inter-beat interval recorder employing two electrodes on the chest for measurement and that can be considered as a semi-professional device (Parak and Korhonen, 2013).

Smartwatches thus collect a magnitude of data, most of which is sufficient for giving their user an overview of their daily activity. However, when being applied to more serious medical and healthcare use cases, the current measurement accuracy of wearable devices remains rather unclear (El-Amrawy and Nounou, 2015; Phan et al., 2015b; Ge et al., 2016; Dooley et al., 2017; Gillinov et al., 2017).

2.2. Heart Rate Variability

HRV measurements are relevant in various applications. For example, the study conducted in (Cichosz et al., 2014) used an ECG to derive HRV for hypoglycemia prediction and detection. However, to reach a broader audience, we propose to use consumer wearable devices such as smartwatches instead of professional ECGs to measure HRV. While some studies show that data from smartwatches was highly accurate when compared with professional ECGs in long-term measurements (Phan et al., 2015a), other previous research observed a discrepancy between measurements from wrist-worn trackers and ECGs (Dooley et al., 2017; Gillinov et al., 2017).

**Resolution: ** HRV can be measured in both the time and frequency domain. According to the Task Force of the European Society of Cardiology (Malik, 1996), short-term recordings (i.e. 2 to 5 minutes) should be assessed with standard deviation of normal-to-normal interval (SDNN) and root mean square of the successive differences (RMSSD) in the time domain. However, RMSSD has been found to be reliable when calculated with a sample of 10, 30 or 60 seconds, which was not always the case for SDNN, or frequency-based measures (Thong et al., 2004; Salahuddin et al., 2007; Nussinovitch et al., 2011; Esco and Flatt, 2014; Baek et al., 2015; Munoz et al., 2015). Ultra-short-term measures of HRV such as RMSSD would thus allow monitoring streams of physiological changes with a relatively high resolution.

HRV** and hypoglycemia: ** HRV has been shown to correlate positively with hypoglycemia (Schächinger et al., 2004), although in a short-term measure (i.e., 5 min), and with continuous glucose monitoring as a valid predictor of hypoglycemia (Cichosz et al., 2014). It is therefore in our interest to explore the possibility of using an ultra-short-term measurement of HRV to predict hypoglycemia.

3. Work-in-progress research

3.1. Problem Statement

Many widespread smartwatches are capable of measuring HRV, however, our analysis has shown that they incur systematic measurement errors leading to inaccurate HRV measurements. We hypothesize that these data quality issues are related to the movement or physical activity of the wearer and the device being unable to properly measure inter-beat intervals during these times.

Smartwatches are nowadays equipped with a range of sensors relevant for health measurement, such as for example, an optical heart sensor, an accelerometer, a compass, or sensors for positioning information. These sensors can potentially provide valuable additional information.

Our aim is to minimize the error of smartwatch HRV measurements by using additional available sensor information for correcting HRV measurements. With our work, we want to improve the quality and reliability of the RMSSD calculated by smartwatches. The basis of these calculations are the inter-beat intervals as measured by those devices.

3.2. Methods

**Devices: ** In our preliminary research, we compare data obtained from a chest-based heart rate monitor (Firstbeat Bodyguard 2) with data from one of the most widespread consumer smartwatches which is equipped with an accelerometer and an optical heart rate sensor capable of measuring inter-beat intervals.

The heart rate monitor allows recording inter-beat intervals by means of a two-lead measurement, thus providing reference HRV data for our investigations. Given the cited related work on smartwatches we consider obtained from the consumer smartwatch to be less accurate. However, the consumer smartwatch provides further data dimensions, which we take into account in order to compensate for the inaccuracies in HRV measurements imposed by inaccurate inter-beat interval data.

For instance, we aim to improve the accuracy of RMSSD calculations by including both inter-beat intervals and movement indices such as orientation and angular velocity data from accelerometer sensors into the corresponding calculations.

**Analysis: ** In a first step, we use explanatory analysis to compare and investigate the differences between the HRV measurements from both devices, and how the differences relate to the wearer movement. Then, in a second step, we apply predictive modeling in order to forecast measurement errors imposed by the described wearer movements.

**Prediction Model: ** In order to estimate the aforementioned measurement error, we make use of a convolutional neural network (CNN). CNNs are a sub-group of neural networks, which are commonly used for classification tasks like object recognition in images (Krizhevsky et al., 2012). In particular, CNNs have also successfully been applied to medical time-series data such as for ECG classification (Zihlmann et al., 2017).

In our model, we frame the obtained data as a time-series task in a supervised learning setting. The aim is to estimate the errors between the consumer smartwatch RMSSDs (samples in the machine learning model) and the reference heart rate monitor RMSSDs (labels in the machine learning model).

3.3. Data

Raw data was obtained from the heart rate monitor via the Firstbeat SPORTS Individual software. From the consumer smartwatch, data was gathered via a custom smartphone app utilizing the Bluetooth streaming capability of the smartwatch.

Data was recorded by a healthy individual wearing both devices simultaneously over a period of 72 hours. After pre-processing, a total of roughly 200,000 observations consisting of heart rate monitor inter-beat interval, consumer smartwatch inter-beat interval and consumer smartwatch three-axis accelerometer data were left. We split the dataset into 80% of the observations for training and 20% for testing.

4. Results

We quantify HRV by RMSSD which is calculated as

[TABLE]

where $N$ is the number of samples within a sliding window of 60 seconds and $IBI_{i}$ is the $i$-th inter-beat interval within those samples.

The error of the measurements between the heart rate monitor and the consumer smartwatch at time $t$ is defined as

[TABLE]

4.1. Explanatory Analysis

We first tested for a tendency to systematic errors in HRV measurements of the consumer smartwatch, which are imposed by external influences such as the wearer movement. As a result, we found a statistically significant positive correlation between accelerator movement and measurement error, $r=0.22$, $p=\leq 0.001$.

4.2. Predictive Analysis

**Error Prediction Model: ** In our approach, we used a deep CNN in order to predict the error of the measurement. Subsequently, the predicted error is added to the RMSSD as calculated from the raw inter-beat intervals from the consumer smartwatch in order to get an approximation of the reference RMSSD calculated from the heart rate monitor data.

**Performance: ** The first experiment yielded an improvement of the root-mean-squared error (RMSE) between heart rate monitor and consumer smartwatch RMSSDs from an initial 48.89 down to 28.50 on the test set. The statistically significant correlation of heart rate monitor and consumer smartwatch RMSSDs in the test set was improved from $r=0.37$, $p=\leq 0.001$ before adjusting to $r=0.58$, $p=\leq 0.001$ after adjusting.

Figure 1 visualizes this reduction of the measurement error in RMSSDs. For the first part of the graph, the RMSSD error is rather low. During this time, the wearer was in bed, which is reflected by low accelerometer movement. Later on, at around 06:30 when the wearer got out of bed, a rise of accelerometer movement and a deterioration of the error in raw RMSSD measurements are observable. Additionally, it can be observed how the addition of the error as predicted by the CNN turns from positive into negative. The sensor data suggests that a larger magnitude of accelerometer movement is related to a larger error in measurement.

5. Conclusion

Prediction of HRV measurements with smartwatches would allow non-invasive continuous monitoring of psychophysiological conditions such as the risk of hypoglycemia. The key benefit of being able to accurately measure HRV with smartwatches would certainly be their popularity and easiness of use in contrast to professional devices. The implemented CNN is capable of reducing the error in consumer smartwatch HRV measurements by taking into account additional motion information in the form of accelerometer data.

**Current Limitations: ** In future research, we will investigate the generalizability of our approach, which needs to be done in a larger-scale study. Furthermore, the restricted accuracy of the heart rate monitor (see (Parak and Korhonen, 2013)) also limits the accuracy of the model evolved on top of its data. We thus aim to conduct a larger-scale study in which we gather reference data with a professional-grade ECG device.

**Outlook: ** We found strong evidence that systematic errors in HRV measurements from the consumer smartwatch can be minimized with the utilization of additional data by the use of neural networks. In the future, we can generate additional data dimensions that potentially carry valuable information for our problem (e.g., the time-shifted difference of accelerometer data) by utilizing methods of feature engineering. While we are aware of that our proposed methods will not be capable of completely eliminating the error in smartwatch HRV measurements, the methods we have shown contribute to more reliable measurements of physiological values such as HRV in smartwatches. Furthermore, even more, widespread smartwatches such as the Apple Watch should be investigated.

Acknowledgements.

This work was part-funded by the Swiss National Science Foundation (SNF), Project 183569.