Self-Supervised Dynamical System Representations for Physiological Time-Series

TL;DR

This paper introduces PULSE, a self-supervised learning framework for physiological time-series that captures underlying system information while filtering out noise, improving representation quality for classification and transfer learning.

Contribution

The paper proposes PULSE, a novel pretraining method leveraging dynamical system structure to enhance physiological time-series representations, with theoretical guarantees and empirical validation.

Findings

PULSE effectively captures system information in synthetic experiments.

PULSE improves class distinction and label efficiency on real datasets.

PULSE enhances transfer learning performance across diverse tasks.

Abstract

The effectiveness of self-supervised learning (SSL) for physiological time series depends on the ability of a pretraining objective to preserve information about the underlying physiological state while filtering out unrelated noise. However, existing strategies are limited due to reliance on heuristic principles or poorly constrained generative tasks. To address this limitation, we propose a pretraining framework that exploits the information structure of a dynamical systems generative model across multiple time-series. This framework reveals our key insight that class identity can be efficiently captured by extracting information about the generative variables related to the system parameters shared across similar time series samples, while noise unique to individual samples should be discarded. Building on this insight, we propose PULSE, a cross-reconstruction-based pretraining objective for physiological time series datasets that explicitly extracts system information while discarding non-transferrable sample-specific ones. We establish theory that provides sufficient conditions for the system information to be recovered, and empirically validate it using a synthetic dynamical systems experiment. Furthermore, we apply our method to diverse real-world datasets, demonstrating that PULSE learns representations that can broadly distinguish semantic classes, increase label efficiency, and improve transfer learning.