Polytomous Explanatory Item Response Models for Item Discrimination: Assessing Negative-Framing Effects in Social-Emotional Learning Surveys

TL;DR

This paper extends explanatory item response models to polytomous data, analyzing how negative framing affects item discrimination in social-emotional learning surveys, revealing significant reductions in discrimination due to negative framing.

Contribution

It introduces a novel approach for modeling item discrimination in polytomous IRT models and demonstrates its application in assessing framing effects in preschool surveys.

Findings

Negative framing reduces item discrimination by about 30% in one survey.

Negative framing predicts significantly lower discrimination in two of four surveys.

The proposed model extends explanatory IRT to polytomous responses.

Abstract

Modeling item parameters as a function of item characteristics has a long history but has generally focused on models for item location. Explanatory item response models for item discrimination are available but rarely used. In this study, we extend existing approaches for modeling item discrimination from dichotomous to polytomous item responses. We illustrate our proposed approach with an application to four social-emotional learning surveys of preschool children to investigate how item discrimination depends on whether an item is positively or negatively framed. Negative framing predicts significantly lower item discrimination on two of the four surveys, and a plausibly causal estimate from a regression discontinuity analysis shows that negative framing reduces discrimination by about 30\% on one survey. We conclude with a discussion of potential applications of explanatory models for item discrimination.