Are Statistical Methods Obsolete in the Era of Deep Learning? A Study of ODE Inverse Problems

TL;DR

This study compares deep learning and statistical methods for solving ODE inverse problems, finding statistical approaches often outperform neural networks in accuracy, robustness, and data efficiency, especially with sparse or noisy data.

Contribution

The paper demonstrates that statistically principled methods remain highly relevant and often superior to neural networks in ODE inverse problems, especially under challenging data conditions.

Findings

Statistical methods achieve lower bias and variance in parameter inference.

Statistical approaches outperform deep learning in out-of-sample predictions.

Statistical methods are more robust to numerical imprecision and data sparsity.

Abstract

In the era of AI, neural networks have become increasingly popular for modeling, inference, and prediction, largely due to their potential for universal approximation. With the proliferation of such deep learning models, a question arises: are leaner statistical methods still relevant? To shed insight on this question, we employ the mechanistic nonlinear ordinary differential equation (ODE) inverse problem as a testbed, using the physics-informed neural network (PINN) as a representative of the deep learning paradigm and manifold-constrained Gaussian process inference (MAGI) as a representative of statistically principled methods. Through case studies involving the SEIR model from epidemiology and the Lorenz model from chaotic dynamics, we demonstrate that statistical methods are far from obsolete, especially when working with sparse and noisy observations. On tasks such as parameter inference and trajectory reconstruction, statistically principled methods consistently achieve lower bias and variance, while using far fewer parameters and requiring less hyperparameter tuning. Statistical methods can also decisively outperform deep learning models on out-of-sample future prediction, where the absence of relevant data often leads overparameterized models astray. Additionally, we find that statistically principled approaches are more robust to accumulation of numerical imprecision and can represent the underlying system more faithfully to the true governing ODEs.