Data transforming augmentation for heteroscedastic models

TL;DR

This paper introduces data transforming augmentation (DTA), a framework that simplifies heteroscedastic models by converting them into homoscedastic ones, leading to faster algorithms and easier posterior sampling.

Contribution

It proposes a novel DTA scheme for heteroscedastic models, demonstrating improved computational efficiency and convergence in linear mixed models and Beta-Binomial models.

Findings

Faster convergence of EM and Gibbs algorithms with DTA

Simpler computations for heteroscedastic models

Enables sampling of marginal posteriors in complex models

Abstract

Data augmentation (DA) turns seemingly intractable computational problems into simple ones by augmenting latent missing data. In addition to computational simplicity, it is now well-established that DA equipped with a deterministic transformation can improve the convergence speed of iterative algorithms such as an EM algorithm or Gibbs sampler. In this article, we outline a framework for the transformation-based DA, which we call data transforming augmentation (DTA), allowing augmented data to be a deterministic function of latent and observed data, and unknown parameters. Under this framework, we investigate a novel DTA scheme that turns heteroscedastic models into homoscedastic ones to take advantage of simpler computations typically available in homoscedastic cases. Applying this DTA scheme to fitting linear mixed models, we demonstrate simpler computations and faster convergence rates of resulting iterative algorithms, compared with those under a non-transformation-based DA scheme. We also fit a Beta-Binomial model using the proposed DTA scheme, which enables sampling approximate marginal posterior distributions that are available only under homoscedasticity. An R package, Rdta, is publicly available at CRAN.