Beyond Observations: Reconstruction Error-Guided Irregularly Sampled Time Series Representation Learning

TL;DR

This paper introduces iTimER, a self-supervised learning framework for irregularly sampled time series that leverages reconstruction error distributions and contrastive learning to improve representation quality.

Contribution

It proposes a novel error distribution modeling and pseudo-observation generation approach for ISTS, enhancing learning without reliance on observed data imputation.

Findings

Outperforms state-of-the-art methods on classification tasks

Improves interpolation accuracy for missing data

Enhances forecasting performance on irregular time series

Abstract

Irregularly sampled time series (ISTS), characterized by non-uniform time intervals with natural missingness, are prevalent in real-world applications. Existing approaches for ISTS modeling primarily rely on observed values to impute unobserved ones or infer latent dynamics. However, these methods overlook a critical source of learning signal: the reconstruction error inherently produced during model training. Such error implicitly reflects how well a model captures the underlying data structure and can serve as an informative proxy for unobserved values. To exploit this insight, we propose iTimER, a simple yet effective self-supervised pre-training framework for ISTS representation learning. iTimER models the distribution of reconstruction errors over observed values and generates pseudo-observations for unobserved timestamps through a mixup strategy between sampled errors and the last available observations. This transforms unobserved timestamps into noise-aware training targets, enabling meaningful reconstruction signals. A Wasserstein metric aligns reconstruction error distributions between observed and pseudo-observed regions, while a contrastive learning objective enhances the discriminability of learned representations. Extensive experiments on classification, interpolation, and forecasting tasks demonstrate that iTimER consistently outperforms state-of-the-art methods under the ISTS setting.