Continuous time recurrent neural networks: overview and application to forecasting blood glucose in the intensive care unit

TL;DR

This paper reviews continuous time recurrent neural networks (CTRNNs), especially neural ODEs, and demonstrates their application to probabilistic blood glucose forecasting in critical care, showing performance improvements over traditional models.

Contribution

It provides an overview of CTRNN architectures and evaluates their effectiveness in irregular time series forecasting within a medical context.

Findings

Neural ODE and neural flow layers improve CTRNN performance.

LSTM and neural ODE architectures achieve comparable results to gradient boosting.

CTRNNs outperform some architectures but are sometimes outperformed by gradient boosted trees.

Abstract

Irregularly measured time series are common in many of the applied settings in which time series modelling is a key statistical tool, including medicine. This provides challenges in model choice, often necessitating imputation or similar strategies. Continuous time autoregressive recurrent neural networks (CTRNNs) are a deep learning model that account for irregular observations through incorporating continuous evolution of the hidden states between observations. This is achieved using a neural ordinary differential equation (ODE) or neural flow layer. In this manuscript, we give an overview of these models, including the varying architectures that have been proposed to account for issues such as ongoing medical interventions. Further, we demonstrate the application of these models to probabilistic forecasting of blood glucose in a critical care setting using electronic medical record and simulated data. The experiments confirm that addition of a neural ODE or neural flow layer generally improves the performance of autoregressive recurrent neural networks in the irregular measurement setting. However, several CTRNN architecture are outperformed by an autoregressive gradient boosted tree model (Catboost), with only a long short-term memory (LSTM) and neural ODE based architecture (ODE-LSTM) achieving comparable performance on probabilistic forecasting metrics such as the continuous ranked probability score (ODE-LSTM: 0.118$\pm$0.001; Catboost: 0.118$\pm$0.001), ignorance score (0.152$\pm$0.008; 0.149$\pm$0.002) and interval score (175$\pm$1; 176$\pm$1).