Cross-Modal Health State Estimation

TL;DR

This paper presents a method for integrating diverse personal data streams and biomedical knowledge to estimate cardiovascular health states, including fitness and genetic traits, from multi-modal data for personalized health insights.

Contribution

It introduces a novel approach to fuse multi-modal data and domain knowledge for continuous health state estimation, specifically targeting cardiovascular health.

Findings

Multi-modal data fusion improves health state accuracy.

Personalized health insights can be derived from diverse data sources.

Experimental results on 24 subjects validate the approach.

Abstract

Individuals create and consume more diverse data about themselves today than any time in history. Sources of this data include wearable devices, images, social media, geospatial information and more. A tremendous opportunity rests within cross-modal data analysis that leverages existing domain knowledge methods to understand and guide human health. Especially in chronic diseases, current medical practice uses a combination of sparse hospital based biological metrics (blood tests, expensive imaging, etc.) to understand the evolving health status of an individual. Future health systems must integrate data created at the individual level to better understand health status perpetually, especially in a cybernetic framework. In this work we fuse multiple user created and open source data streams along with established biomedical domain knowledge to give two types of quantitative state estimates of cardiovascular health. First, we use wearable devices to calculate cardiorespiratory fitness (CRF), a known quantitative leading predictor of heart disease which is not routinely collected in clinical settings. Second, we estimate inherent genetic traits, living environmental risks, circadian rhythm, and biological metrics from a diverse dataset. Our experimental results on 24 subjects demonstrate how multi-modal data can provide personalized health insight. Understanding the dynamic nature of health status will pave the way for better health based recommendation engines, better clinical decision making and positive lifestyle changes.