SeqLink: A Robust Neural-ODE Architecture for Modelling Partially Observed Time Series

TL;DR

SeqLink introduces a neural-ODE architecture that enhances time series modeling robustness by leveraging multiple data samples and sequence links, outperforming existing methods especially on sparse and intermittent data.

Contribution

The paper proposes SeqLink, a novel neural architecture that improves the modeling of partially observed time series by utilizing relationships between samples to define hidden states.

Findings

SeqLink outperforms state-of-the-art models on synthetic and real-world datasets.

The model effectively handles long sequences and data sparsity.

SeqLink improves the modeling of intermittent time series.

Abstract

Ordinary Differential Equations (ODE) based models have become popular as foundation models for solving many time series problems. Combining neural ODEs with traditional RNN models has provided the best representation for irregular time series. However, ODE-based models typically require the trajectory of hidden states to be defined based on either the initial observed value or the most recent observation, raising questions about their effectiveness when dealing with longer sequences and extended time intervals. In this article, we explore the behaviour of the ODE models in the context of time series data with varying degrees of sparsity. We introduce SeqLink, an innovative neural architecture designed to enhance the robustness of sequence representation. Unlike traditional approaches that solely rely on the hidden state generated from the last observed value, SeqLink leverages ODE latent representations derived from multiple data samples, enabling it to generate robust data representations regardless of sequence length or data sparsity level. The core concept behind our model is the definition of hidden states for the unobserved values based on the relationships between samples (links between sequences). Through extensive experiments on partially observed synthetic and real-world datasets, we demonstrate that SeqLink improves the modelling of intermittent time series, consistently outperforming state-of-the-art approaches.