Neural Chronos ODE: Unveiling Temporal Patterns and Forecasting Future and Past Trends in Time Series Data

TL;DR

Neural Chronos ODE (Neural CODE) is a novel continuous-time neural network architecture that models system dynamics in both forward and backward directions, outperforming Neural ODEs and RNN-based models in time series forecasting and imputation tasks.

Contribution

The paper introduces Neural CODE, a new ODE-based neural network architecture that effectively captures bidirectional temporal dynamics and enhances performance over existing models.

Findings

Neural CODE outperforms Neural ODE in learning spiral dynamics.

Bidirectional architectures like CODE-BiRNN outperform unidirectional models.

Proposed models converge faster and achieve better accuracy on real-world data.

Abstract

This work introduces Neural Chronos Ordinary Differential Equations (Neural CODE), a deep neural network architecture that fits a continuous-time ODE dynamics for predicting the chronology of a system both forward and backward in time. To train the model, we solve the ODE as an initial value problem and a final value problem, similar to Neural ODEs. We also explore two approaches to combining Neural CODE with Recurrent Neural Networks by replacing Neural ODE with Neural CODE (CODE-RNN), and incorporating a bidirectional RNN for full information flow in both time directions (CODE-BiRNN), and variants with other update cells namely GRU and LSTM: CODE-GRU, CODE-BiGRU, CODE-LSTM, CODE-BiLSTM. Experimental results demonstrate that Neural CODE outperforms Neural ODE in learning the dynamics of a spiral forward and backward in time, even with sparser data. We also compare the performance of CODE-RNN/-GRU/-LSTM and CODE-BiRNN/-BiGRU/-BiLSTM against ODE-RNN/-GRU/-LSTM on three real-life time series data tasks: imputation of missing data for lower and higher dimensional data, and forward and backward extrapolation with shorter and longer time horizons. Our findings show that the proposed architectures converge faster, with CODE-BiRNN/-BiGRU/-BiLSTM consistently outperforming the other architectures on all tasks.