VCR: Learning Valid Contextual Representation for Incomplete Wearable Signals

TL;DR

VCR is a self-supervised framework that learns robust, valid representations from multimodal wearable signals, effectively handling missing modalities without hallucinating unobservable details, thereby improving health monitoring robustness.

Contribution

VCR introduces an orthogonal tokenizer and missing-aware mixture-of-experts to disentangle shared and modality-specific information, enhancing robustness to missing data in wearable health signals.

Findings

VCR outperforms baselines in health monitoring tasks with missing modalities.

VCR maintains high performance even with multiple missing sensors.

VCR reduces hallucination of unobservable modality-specific details.

Abstract

Wearable devices enable continuous health monitoring from multimodal signals, but real-world deployment is hindered by limited labeled data and pervasive sensor incompleteness. While large-scale self-supervised pretraining reduces label dependence, most existing methods assume full modality availability. Current approaches for handling modality missingness often reconstruct entire absent signals, which can encourage hallucinating modality-specific details that are not inferable from the observed sensor signals and degrade robustness. We propose VCR, a self-supervised framework that learns to extract valid representations robust to modality missingness. VCR employs an orthogonal tokenizer to enforce strict orthogonal disentanglement by rectifying latent manifolds and applying a geometric projection, separating each modality into shared semantics and modality-specific residuals. This design preserves complete information integrity while serving as a structural foundation for robust learning under modality missingness. The resulting tokens are processed by a missing-aware mixture-of-experts backbone that adapts to varying patterns of modality availability. By constraining the objective to reconstruct only the shared components of missing modalities, VCR effectively mitigates hallucinations of non-inferable modality-specific details. Across multiple health monitoring tasks, VCR consistently improves performance and robustness under full, single-missing, and multiple-missing modality settings compared with strong supervised and self-supervised baselines.