Bayesian comparison of latent variable models: Conditional vs marginal likelihoods

TL;DR

This paper clarifies the distinction between conditional and marginal likelihoods in Bayesian models with latent variables, illustrating how this affects model comparison criteria like DIC and WAIC, especially in psychometric contexts.

Contribution

It provides a detailed analysis of the differences between conditional and marginal likelihood-based criteria and offers practical recommendations for their application in latent variable models.

Findings

Marginal WAIC aligns with leave-one-cluster out cross-validation.

Conditional WAIC corresponds to leave-one-unit out cross-validation.

The distinction impacts model comparison and predictive inference in latent variable models.

Abstract

Typical Bayesian methods for models with latent variables (or random effects) involve directly sampling the latent variables along with the model parameters. In high-level software code for model definitions (using, e.g., BUGS, JAGS, Stan), the likelihood is therefore specified as conditional on the latent variables. This can lead researchers to perform model comparisons via conditional likelihoods, where the latent variables are considered model parameters. In other settings, however, typical model comparisons involve marginal likelihoods where the latent variables are integrated out. This distinction is often overlooked despite the fact that it can have a large impact on the comparisons of interest. In this paper, we clarify and illustrate these issues, focusing on the comparison of conditional and marginal Deviance Information Criteria (DICs) and Watanabe-Akaike Information Criteria (WAICs) in psychometric modeling. The conditional/marginal distinction corresponds to whether the model should be predictive for the clusters that are in the data or for new clusters (where "clusters" typically correspond to higher-level units like people or schools). Correspondingly, we show that marginal WAIC corresponds to leave-one-cluster out (LOcO) cross-validation, whereas conditional WAIC corresponds to leave-one-unit out (LOuO). These results lead to recommendations on the general application of the criteria to models with latent variables.