Mixed-type multivariate response regression with covariance estimation

TL;DR

This paper introduces a flexible multivariate response regression method for mixed data types, leveraging a latent normal model with a novel scalable estimation algorithm, improving prediction and inference in complex datasets.

Contribution

It develops a new latent normal-based model for mixed-type responses with a scalable approximate likelihood estimation algorithm, avoiding restrictive parametric assumptions.

Findings

Better prediction accuracy than separate models

Effective covariance estimation for mixed data

Successful application to biomedical and genomic data

Abstract

We propose a new method for multivariate response regression and covariance estimation when elements of the response vector are of mixed types, for example some continuous and some discrete. Our method is based on a model which assumes the observable mixed-type response vector is connected to a latent multivariate normal response linear regression through a link function. We explore the properties of this model and show its parameters are identifiable under reasonable conditions. We impose no parametric restrictions on the covariance of the latent normal other than positive definiteness, thereby avoiding assumptions about unobservable variables which can be difficult to verify in practice. To accommodate this generality, we propose a novel algorithm for approximate maximum likelihood estimation that works "off-the-shelf" with many different combinations of response types, and which scales well in the dimension of the response vector. Our method typically gives better predictions and parameter estimates than fitting separate models for the different response types and allows for approximate likelihood ratio testing of relevant hypotheses such as independence of responses. The usefulness of the proposed method is illustrated in simulations; and one biomedical and one genomic data example.