Going NUTS with ADVI: Exploring various Bayesian Inference techniques with Facebook Prophet

TL;DR

This paper reimplements Facebook Prophet in PyMC to enable flexible Bayesian inference, compares various inference methods, and discusses their efficiency and convergence on time-series forecasting tasks.

Contribution

We developed a PyMC-based Prophet implementation allowing for diverse Bayesian inference techniques and detailed comparison of their performance.

Findings

Full MCMC, MAP, and Variational inference methods evaluated.

Analysis of sampling, convergence, and forecasting metrics.

Discussion of computational efficiency and potential issues.

Abstract

Since its introduction, Facebook Prophet has attracted positive attention from both classical statisticians and the Bayesian statistics community. The model provides two built-in inference methods: maximum a posteriori estimation using the L-BFGS-B algorithm, and Markov Chain Monte Carlo (MCMC) sampling via the No-U-Turn Sampler (NUTS). While exploring various time-series forecasting problems using Bayesian inference with Prophet, we encountered limitations stemming from the inability to apply alternative inference techniques beyond those provided by default. Additionally, the fluent API design of Facebook Prophet proved insufficiently flexible for implementing our custom modeling ideas. To address these shortcomings, we developed a complete reimplementation of the Prophet model in PyMC, which enables us to extend the base model and evaluate and compare multiple Bayesian inference methods. In this paper, we present our PyMC-based implementation and analyze in detail the implementation of different Bayesian inference techniques. We consider full MCMC techniques, MAP estimation and Variational inference techniques on a time-series forecasting problem. We discuss in details the sampling approach, convergence diagnostics, forecasting metrics as well as their computational efficiency and detect possible issues which will be addressed in our future work.