EMIT- Event-Based Masked Auto Encoding for Irregular Time Series

TL;DR

This paper introduces EMIT, a novel self-supervised pretraining method for irregular time series that uses event-based masking to improve model understanding of variable measurement intervals, especially in healthcare data.

Contribution

EMIT is the first to apply event-based masking in latent space for irregular time series, enhancing model robustness and performance over existing methods.

Findings

Outperforms existing models on MIMIC-III and PhysioNet datasets.

Effectively captures variability and timing in irregular measurements.

Improves downstream task accuracy in healthcare applications.

Abstract

Irregular time series, where data points are recorded at uneven intervals, are prevalent in healthcare settings, such as emergency wards where vital signs and laboratory results are captured at varying times. This variability, which reflects critical fluctuations in patient health, is essential for informed clinical decision-making. Existing self-supervised learning research on irregular time series often relies on generic pretext tasks like forecasting, which may not fully utilise the signal provided by irregular time series. There is a significant need for specialised pretext tasks designed for the characteristics of irregular time series to enhance model performance and robustness, especially in scenarios with limited data availability. This paper proposes a novel pretraining framework, EMIT, an event-based masking for irregular time series. EMIT focuses on masking-based reconstruction in the latent space, selecting masking points based on the rate of change in the data. This method preserves the natural variability and timing of measurements while enhancing the model's ability to process irregular intervals without losing essential information. Extensive experiments on the MIMIC-III and PhysioNet Challenge datasets demonstrate the superior performance of our event-based masking strategy. The code has been released at https://github.com/hrishi-ds/EMIT.