Bayesian inference for a covariance matrix

TL;DR

This paper evaluates how different Bayesian priors for covariance matrices affect posterior inference, highlighting the limitations of the inverse Wishart prior and exploring alternative distributions and strategies.

Contribution

It provides a comparative analysis of prior choices for covariance matrices, including the inverse Wishart, scaled inverse Wishart, and separate priors, assessing their impact on inference.

Findings

Inverse Wishart can be too restrictive and impose prior relationships.

Scaled inverse Wishart offers more flexibility but retains some prior relationships.

Separate priors eliminate relationships but are computationally slower.

Abstract

Covariance matrix estimation arises in multivariate problems including multivariate normal sampling models and regression models where random effects are jointly modeled, e.g. random-intercept, random-slope models. A Bayesian analysis of these problems requires a prior on the covariance matrix. Here we assess, through a simulation study and a real data set, the impact this prior choice has on posterior inference of the covariance matrix. Inverse Wishart distribution is the natural choice for a covariance matrix prior because its conjugacy on normal model and simplicity, is usually available in Bayesian statistical software. However inverse Wishart distribution presents some undesirable properties from a modeling point of view. It can be too restrictive because assume the same amount of prior information about every variance parameters and, more important, it shows a prior relationship between the variances and correlations. Some alternatives distributions has been proposed. The scaled inverse Wishart distribution, which give more flexibility on the variance priors conserving the conjugacy property but does not eliminate the prior relationship between variances and correlations. Secondly, it is possible to fit separate priors for individual correlations and standard deviations. This strategy eliminates any prior relationship within the covariance matrix parameters, but it is not conjugate and therefore computationally slow.