Data augmentation for non-Gaussian regression models using variance-mean mixtures

TL;DR

This paper introduces a data-augmentation approach based on normal variance-mean mixtures for non-Gaussian regression models, enhancing existing methods and enabling faster algorithms for diverse applications.

Contribution

It generalizes normal variance mixture theory to a broader class of models and incorporates quasi-Newton acceleration for improved computational efficiency.

Findings

Effective data-augmentation scheme for non-Gaussian models

Application to sparse quantile and binary logistic regression

Significant speed-up with quasi-Newton acceleration

Abstract

We use the theory of normal variance-mean mixtures to derive a data-augmentation scheme for a class of common regularization problems. This generalizes existing theory on normal variance mixtures for priors in regression and classification. It also allows variants of the expectation-maximization algorithm to be brought to bear on a wider range of models than previously appreciated. We demonstrate the method on several examples, including sparse quantile regression and binary logistic regression. We also show that quasi-Newton acceleration can substantially improve the speed of the algorithm without compromising its robustness.