Flexible online multivariate regression with variational Bayes and the matrix-variate Dirichlet process

TL;DR

This paper develops fast online variational Bayes algorithms for a flexible multivariate regression model using the matrix-variate Dirichlet process, enabling efficient distributional analysis of responses with covariates.

Contribution

It introduces online variational Bayes methods for the matrix-variate Dirichlet process regression, improving computational efficiency over existing MCMC and batch approaches.

Findings

Online variational Bayes outperforms MCMC in speed.

The method maintains comparable accuracy to batch variational approaches.

Efficiently models complex distributional changes in multivariate responses.

Abstract

Flexible regression methods where interest centres on the way that the whole distribution of a response vector changes with covariates are very useful in some applications. A recently developed technique in this regard uses the matrix-variate Dirichlet process as a prior for a mixing distribution on a coefficient in a multivariate linear regression model. The method is attractive, particularly in the multivariate setting, for the convenient way that it allows for borrowing strength across different component regressions and for its computational simplicity and tractability. The purpose of the present article is to develop fast online variational Bayes approaches to fitting this model and to investigate how they perform compared to MCMC and batch variational methods in a number of scenarios.