Stochastic Variational Inference for Structured Additive Distributional Regression

TL;DR

This paper introduces a stochastic variational inference method for structured additive distributional regression, enabling efficient Bayesian inference for complex models with multiple parameters and smoothing components.

Contribution

It proposes two novel strategies for constructing multivariate Gaussian variational distributions and two approaches for estimating smoothing parameters within this framework.

Findings

Benchmarking shows competitive performance against state-of-the-art methods.

Validation with MCMC confirms the accuracy of the variational posterior estimates.

The methods effectively handle complex models with multiple parameters and uncertainty quantification.

Abstract

In structured additive distributional regression, the conditional distribution of the response variables given the covariate information and the vector of model parameters is modelled using a P-parametric probability density function where each parameter is modelled through a linear predictor and a bijective response function that maps the domain of the predictor into the domain of the parameter. We present a method to perform inference in structured additive distributional regression using stochastic variational inference. We propose two strategies for constructing a multivariate Gaussian variational distribution to estimate the posterior distribution of the regression coefficients. The first strategy leverages covariate information and hyperparameters to learn both the location vector and the precision matrix. The second strategy tackles the complexity challenges of the first by initially assuming independence among all smooth terms and then introducing correlations through an additional set of variational parameters. Furthermore, we present two approaches for estimating the smoothing parameters. The first treats them as free parameters and provides point estimates, while the second accounts for uncertainty by applying a variational approximation to the posterior distribution. Our model was benchmarked against state-of-the-art competitors in logistic and gamma regression simulation studies. Finally, we validated our approach by comparing its posterior estimates to those obtained using Markov Chain Monte Carlo on a dataset of patents from the biotechnology/pharmaceutics and semiconductor/computer sectors.