Efficient Multi-View Fusion and Flexible Adaptation to View Missing in Cardiovascular System Signals

TL;DR

This paper introduces a view-centric transformer and multitask autoencoder for multi-view fusion of cardiovascular signals, addressing asynchronous data, view heterogeneity, and missing views with minimal fine-tuning, improving health monitoring tasks.

Contribution

The paper proposes novel view-centric models and prompt techniques for flexible adaptation to missing views in multi-view cardiovascular data, enhancing robustness and efficiency.

Findings

Outperforms existing methods in atrial fibrillation detection, blood pressure estimation, and sleep staging.

Prompt tuning requires less than 3% of model data for adaptation.

Effectively handles incomplete view data without full retraining.

Abstract

The progression of deep learning and the widespread adoption of sensors have facilitated automatic multi-view fusion (MVF) about the cardiovascular system (CVS) signals. However, prevalent MVF model architecture often amalgamates CVS signals from the same temporal step but different views into a unified representation, disregarding the asynchronous nature of cardiovascular events and the inherent heterogeneity across views, leading to catastrophic view confusion. Efficient training strategies specifically tailored for MVF models to attain comprehensive representations need simultaneous consideration. Crucially, real-world data frequently arrives with incomplete views, an aspect rarely noticed by researchers. Thus, the View-Centric Transformer (VCT) and Multitask Masked Autoencoder (M2AE) are specifically designed to emphasize the centrality of each view and harness unlabeled data to achieve superior fused representations. Additionally, we systematically define the missing-view problem for the first time and introduce prompt techniques to aid pretrained MVF models in flexibly adapting to various missing-view scenarios. Rigorous experiments involving atrial fibrillation detection, blood pressure estimation, and sleep staging-typical health monitoring tasks-demonstrate the remarkable advantage of our method in MVF compared to prevailing methodologies. Notably, the prompt technique requires finetuning less than 3% of the entire model's data, substantially fortifying the model's resilience to view missing while circumventing the need for complete retraining. The results demonstrate the effectiveness of our approaches, highlighting their potential for practical applications in cardiovascular health monitoring. Codes and models are released at URL.