Examinees' Rapid-Guessing Patterns in Computerized Adaptive Testing for Interim Assessment: From Hierarchical Clustering

TL;DR

This study uses hierarchical clustering to analyze nuanced rapid-guessing patterns among students in computerized adaptive testing for interim assessment, revealing how various factors influence examinee behavior.

Contribution

It introduces hierarchical clustering as a novel method to analyze rapid-guessing patterns in CATs for interim assessment, providing detailed insights into examinee behaviors.

Findings

Rapid-guessing patterns vary across item positions and content domains.

Examinee clusters show distinct rapid-guessing behaviors.

Patterns differ by grade level and overall guessing levels.

Abstract

Interim assessment is frequently administered via computerized adaptive testing (CAT), offering direct support to teaching and learning. This study attempted to fill a vital knowledge gap about the nuanced landscape of examinees' rapid-guessing patterns in CAT in the interim assessment context. We analyzed a sample of 146,519 examinees in Grades 1-8 who participated in a widely used CAT, using hierarchical clustering, a robust data science methodology for uncovering insights in data. We found that examinees' rapid-guessing patterns varied across item positions, content domains, chronological grades, examinee clusters, and examinees' overall rapid-guessing level on the test, suggesting a nuanced interplay between testing features and examinees' behavior. Our study contributes to the literature on rapid guessing in CATs for interim assessment, offering a comprehensive and nuanced pattern analysis and demonstrating the application of hierarchical clustering to process data analysis in testing.