Posterior predictive checks for the detection of extreme response style

TL;DR

This paper explores using Bayesian posterior predictive checks to detect extreme response styles in surveys without needing extra questionnaires or complex models.

Contribution

The paper introduces Bayesian posterior predictive checks as a novel method for detecting extreme response styles in questionnaire data.

Findings

Posterior predictive checks can detect extreme response styles at both group and individual levels.

The method does not require additional questionnaires or assumptions about the nature of extreme response styles.

Various PPCs tailored to ERS are proposed and tested in an empirical example.

Abstract

Extreme response style (ERS), the tendency of participants to select extreme item categories regardless of the item content, has frequently been found to decrease the validity of Likert-type questionnaire results (e.g., Moors, European Journal of Work and Organizational Psychology, 21, 271–298, 2012). For this reason, detecting ERS at both the group and individual levels is of paramount importance. While various approaches to detecting ERS exist, these may conflate ERS with the trait of interest, require additional questionnaires to be administered, or require the use of mixture or multidimensional IRT models. As an alternative approach to detecting ERS, Bayesian posterior predictive checks (PPCs) may be a viable option. Posterior predictive checking offers a highly customizable framework for detecting model misfit, which can be directly applied to frequently used unidimensional IRT models. Critically, the use of PPCs to detect ERS does not require strong assumptions regarding the nature of ERS, such as ERS being a continuous dimension or a categorical trait. In this paper, we thus apply PPCs to a generalized partial credit model to detect model misfit related to ERS on both the group and person levels. We propose various possible PPCs tailored to ERS, which are illustrated in an empirical example, and their performance in detecting ERS is examined under various conditions. Suggestions for practical applications are provided, and avenues for future research are explored.

Figures