Model Assessment for a Generalised Bayesian Structural Equation Model

TL;DR

This paper introduces a new model assessment approach for Bayesian structural equation models that improves out-of-sample predictive evaluation using scoring rules, cross-validation, and informative priors, applicable to various data types.

Contribution

It develops a comprehensive assessment framework for BSEM that overcomes limitations of posterior predictive p-values, incorporating scoring rules, cross-validation, and approximate zero priors.

Findings

Effective out-of-sample predictive performance monitoring

Successful application to real datasets on personality and nicotine dependence

Enhanced model support investigation with new assessment tools

Abstract

The paper proposes a novel model assessment paradigm aiming to address shortcoming of posterior predictive $p-$values, which provide the default metric of fit for Bayesian structural equation modelling (BSEM). The model framework of the paper focuses on the approximate zero approach, according to which parameters that would before set to zero (e.g. factor loadings) are now formulated to be approximate zero via informative priors (Muthen and Asparouhov, 2012). The introduced model assessment procedure monitors the out-of-sample predictive performance of the fitted model, and together with a list of guidelines we provide, one can investigate whether the hypothesised model is supported by the data. We incorporate scoring rules and cross-validation to supplement existing model assessment metrics for Bayesian SEM. The proposed tools can be applied to models for both categorical and continuous data. The modelling of categorical and non-normally distributed continuous data is facilitated with the introduction of an item-individual random effect that can also be used for outlier detection. We study the performance of the proposed methodology via simulations. The factor model for continuous and binary data is fitted to data on the `Big-5' personality scale and the Fagerstrom test for nicotine dependence respectively.