Marginalization Consistent Probabilistic Forecasting of Irregular Time Series via Mixture of Separable flows

TL;DR

This paper introduces MOSES, a probabilistic forecasting model for irregular time series that guarantees marginalization consistency, enabling accurate marginal predictions and broad probabilistic querying.

Contribution

MOSES is the first model to ensure marginalization consistency in probabilistic forecasting of irregular time series, combining Gaussian processes with separable normalizing flows.

Findings

MOSES outperforms marginalization consistent baselines in marginal predictions.

MOSES achieves competitive joint prediction accuracy.

MOSES enables accurate probabilistic queries beyond joint distributions.

Abstract

Probabilistic forecasting models for joint distributions of targets in irregular time series with missing values are a heavily under-researched area in machine learning, with, to the best of our knowledge, only two Models have been researched so far: The Gaussian Process Regression model, and ProFITi. While ProFITi, thanks to using multivariate normalizing flows, is very expressive, leading to better predictive performance, it suffers from marginalization inconsistency: It does not guarantee that the marginal distributions of a subset of variables in its predictive distributions coincide with the directly predicted distributions of these variables. When asked to directly predict marginal distributions, they are often vastly inaccurate. We propose MOSES (Marginalization Consistent Mixture of Separable Flows), a model that parametrizes a stochastic process through a mixture of several latent multivariate Gaussian Processes combined with separable univariate Normalizing Flows. In particular, MOSES can be analytically marginalized, allowing it to directly answer a wider range of probabilistic queries than most competitors. Experiments on four datasets show that MOSES achieves both accurate joint and marginal predictions, surpassing all other marginalization consistent baselines, while only trailing slightly behind ProFITi in joint prediction, but vastly superior when predicting marginal distributions.