Joint Assessment of the Differential Item Functioning and Latent Trait Dimensionality of Students' National Tests

TL;DR

This study evaluates the assumptions of unidimensionality and absence of Differential Item Functioning in Italian national student tests using advanced multidimensional IRT models, revealing DIF effects and dimensionality issues.

Contribution

It introduces an extended multidimensional latent class IRT model incorporating DIF and hierarchical clustering to assess test dimensionality and fairness.

Findings

DIF effects are present in both tests.

Unidimensionality is rejected for the Italian Test.

Mathematics Test is reasonably unidimensional.

Abstract

Within the educational context, students' assessment tests are routinely validated through Item Response Theory (IRT) models which assume unidimensionality and absence of Differential Item Functioning (DIF). In this paper, we investigate if such assumptions hold for two national tests administered in Italy to middle school students in June 2009: the Italian Test and the Mathematics Test. To this aim, we rely on an extended class of multidimensional latent class IRT models characterised by: (i) a two-parameter logistic parameterisation for the conditional probability of a correct response, (ii) latent traits represented through a random vector with a discrete distribution, and (iii) the inclusion of (uniform) DIF to account for students' gender and geographical area. A classification of the items into unidimensional groups is also proposed and represented by a dendrogram, which is obtained from a hierarchical clustering algorithm. The results provide evidence for DIF effects for both Tests. Besides, the assumption of unidimensionality is strongly rejected for the Italian Test, whereas it is reasonable for the Mathematics Test.