Uncertainty quantification with approximate variational learning for wearable photoplethysmography prediction tasks

TL;DR

This paper explores uncertainty quantification methods like Monte Carlo Dropout and Variational Online Newton to improve trustworthiness in deep learning models for PPG-based cardiac health assessment, emphasizing hyperparameter tuning.

Contribution

It introduces scalable uncertainty quantification techniques for PPG prediction tasks and analyzes how hyperparameters affect model performance and uncertainty calibration.

Findings

Hyperparameters significantly influence predictive accuracy and uncertainty quality.

Model stochasticity impacts the proportion of aleatoric uncertainty.

Calibration quality varies across predicted classes, requiring thorough evaluation.

Abstract

Photoplethysmography (PPG) signals encode information about relative changes in blood volume that can be used to assess various aspects of cardiac health non-invasively, e.g.\ to detect atrial fibrillation (AF) or predict blood pressure (BP). Deep networks are well-equipped to handle the large quantities of data acquired from wearable measurement devices. However, they lack interpretability and are prone to overfitting, leaving considerable risk for poor performance on unseen data and misdiagnosis. Here, we describe the use of two scalable uncertainty quantification techniques: Monte Carlo Dropout and the recently proposed Improved Variational Online Newton. These techniques are used to assess the trustworthiness of models trained to perform AF classification and BP regression from raw PPG time series. We find that the choice of hyperparameters has a considerable effect on the predictive performance of the models and on the quality and composition of predicted uncertainties. E.g. the stochasticity of the model parameter sampling determines the proportion of the total uncertainty that is aleatoric, and has varying effects on predictive performance and calibration quality dependent on the chosen uncertainty quantification technique and the chosen expression of uncertainty. We find significant discrepancy in the quality of uncertainties over the predicted classes, emphasising the need for a thorough evaluation protocol that assesses local and adaptive calibration. This work suggests that the choice of hyperparameters must be carefully tuned to balance predictive performance and calibration quality, and that the optimal parameterisation may vary depending on the chosen expression of uncertainty.