Emergent Symbolic Structure in Health Foundation Models: Extraction, Alignment, and Cross-Modal Transfer

TL;DR

This paper introduces a post-training framework to interpret, align, and transfer health foundation model embeddings across modalities, revealing a shared symbolic structure that preserves physiological information.

Contribution

The authors propose a novel method to decompose and align frozen embeddings into interpretable symbols, enabling effective cross-modal transfer without retraining.

Findings

Symbols associate with health conditions and physiological attributes.

Cross-modal transfer retains over 95% of in-domain performance.

Alignment recovers a shared low-dimensional physiological subspace.

Abstract

Health foundation models (FMs) learn useful representations from wearable sensors, but interpreting what they encode and transferring that knowledge across modalities after training remains difficult. We present a post-training framework that decomposes frozen embeddings into interpretable directions, referred to as symbols, and use these symbols to align the embedding spaces without retraining. We evaluate the framework on three FMs for photoplethysmography (PPG) and accelerometer data, independently pretrained on ~20M minutes of unlabeled data from ~172K participants, and analyzed on a held-out cohort of 30K subjects. We find that extracted symbols associate selectively with health conditions and physiological attributes, and these associations are partially shared across modalities and architectures. Cross-modal transfer via symbols retains more than 95% of in-domain performance, is nearly symmetric across domain directions, and saturates with limited paired data, together indicating that alignment recovers a shared low-dimensional subspace rich in physiological information. Overall, these results suggest that health FM embeddings contain an interpretable symbolic organization that is shared across modalities and supports cross-domain transfer without joint training.