On the balance between the training time and interpretability of neural ODE for time series modelling

TL;DR

This paper explores the trade-off between interpretability and training time in neural ODEs for time series modeling, proposing a simplified linear approach that balances these aspects.

Contribution

It introduces a method to reduce neural ODEs to linear forms and combines neural networks with ODE systems for more practical time series analysis.

Findings

Neural ODE complexity exceeds traditional models.

Eigenanalysis of the operator is challenging for large systems.

Linear reduction of neural ODEs offers interpretability with manageable training time.

Abstract

Most machine learning methods are used as a black box for modelling. We may try to extract some knowledge from physics-based training methods, such as neural ODE (ordinary differential equation). Neural ODE has advantages like a possibly higher class of represented functions, the extended interpretability compared to black-box machine learning models, ability to describe both trend and local behaviour. Such advantages are especially critical for time series with complicated trends. However, the known drawback is the high training time compared to the autoregressive models and long-short term memory (LSTM) networks widely used for data-driven time series modelling. Therefore, we should be able to balance interpretability and training time to apply neural ODE in practice. The paper shows that modern neural ODE cannot be reduced to simpler models for time-series modelling applications. The complexity of neural ODE is compared to or exceeds the conventional time-series modelling tools. The only interpretation that could be extracted is the eigenspace of the operator, which is an ill-posed problem for a large system. Spectra could be extracted using different classical analysis methods that do not have the drawback of extended time. Consequently, we reduce the neural ODE to a simpler linear form and propose a new view on time-series modelling using combined neural networks and an ODE system approach.