Using Differentiable Programming for Flexible Statistical Modeling

TL;DR

This paper demonstrates how differentiable programming can be effectively used for flexible statistical modeling, enabling quick prototyping and optimization in complex real-world applications like COVID-19 demand prediction.

Contribution

The paper introduces the application of differentiable programming to statistical modeling, showcasing its utility beyond deep learning for rapid model development and optimization.

Findings

Differentiable programming enables gradient-based optimization of statistical models.

A COVID-19 demand prediction model outperformed simpler benchmarks.

The approach facilitates quick prototyping under data quality challenges.

Abstract

Differentiable programming has recently received much interest as a paradigm that facilitates taking gradients of computer programs. While the corresponding flexible gradient-based optimization approaches so far have been used predominantly for deep learning or enriching the latter with modeling components, we want to demonstrate that they can also be useful for statistical modeling per se, e.g., for quick prototyping when classical maximum likelihood approaches are challenging or not feasible. In an application from a COVID-19 setting, we utilize differentiable programming to quickly build and optimize a flexible prediction model adapted to the data quality challenges at hand. Specifically, we develop a regression model, inspired by delay differential equations, that can bridge temporal gaps of observations in the central German registry of COVID-19 intensive care cases for predicting future demand. With this exemplary modeling challenge, we illustrate how differentiable programming can enable simple gradient-based optimization of the model by automatic differentiation. This allowed us to quickly prototype a model under time pressure that outperforms simpler benchmark models. We thus exemplify the potential of differentiable programming also outside deep learning applications, to provide more options for flexible applied statistical modeling.