Mapping unobserved item-respondent interactions: A latent space item response model with interaction map

TL;DR

This paper introduces a latent space item response model that maps unobserved interactions between respondents and items, allowing for better diagnostics and understanding of heterogeneity in response data.

Contribution

The paper proposes a novel latent space model for item response analysis that captures unobserved heterogeneity and interaction maps, improving diagnostic insights.

Findings

The model effectively detects unobserved respondent-item interactions.

Simulation results validate the model's ability to uncover hidden heterogeneity.

Empirical analysis demonstrates practical usefulness in educational assessments.

Abstract

Classic item response models assume that all items with the same difficulty have the same response probability among all respondents with the same ability. These assumptions, however, may very well be violated in practice, and it is not straightforward to assess whether these assumptions are violated, because neither the abilities of respondents nor the difficulties of items are observed. An example is an educational assessment where unobserved heterogeneity is present, arising from unobserved variables such as cultural background and upbringing of students, the quality of mentorship and other forms of emotional and professional support received by students, and other unobserved variables that may affect response probabilities. To address such violations of assumptions, we introduce a novel latent space model which assumes that both items and respondents are embedded in an unobserved metric space, with the probability of a correct response decreasing as a function of the distance between the respondent's and the item's position in the latent space. The resulting latent space approach provides an interaction map that represents interactions of respondents and items, and helps derive insightful diagnostic information on items as well as respondents. In practice, such interaction maps enable teachers to detect students from underrepresented groups who need more support than other students. We provide empirical evidence to demonstrate the usefulness of the proposed latent space approach, along with simulation results.