A Two-Step Method Based on lz* for Identifying Effortful Respondents
Yilan Chen, Yue Liu, Hongyun Liu
TL;DR
This paper introduces a two-step method to improve the accuracy of identifying effortful respondents in educational assessments.
Contribution
The novel approach combines data mining with the lz* statistic to enhance item parameter estimation and respondent identification.
Findings
Using K-means clustering improves the accuracy of item parameter estimates.
The two-step method enhances lz* performance in identifying effortful respondents under high non-effort severity.
Abstract
The likelihood-based person-fit statistic, lz*, is commonly used in educational assessments to distinguish between respondents who are putting in effort and those who are not. However, lz* depends on the estimated item parameters. Item parameter estimates based on data containing non-effortful respondents are biased, thereby undermining the strength of lz*. To address this issue, we propose a two-step method that leverages data mining techniques to obtain more accurate item parameter estimates and then uses them to compute lz*. The results show that the estimates based on the effortful group identified by K-means are more accurate, which improves the performance of lz* in terms of the precision of identifying effortful respondents when non-effort severity is high.
Genes, proteins, chemicals, diseases, species, mutations and cell lines named across the full text — each resolved to its canonical identifier and authoritative record.
Click any figure to enlarge with its caption.
Figure 1
Figure 2
Figure 3
Figure 4
Figure 5
Figure 6
Figure 7
Figure 8
Figure 9
Figure 10
Figure 11
Figure 12
Figure 13
Figure 14Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsPsychometric Methodologies and Testing · Survey Methodology and Nonresponse · Survey Sampling and Estimation Techniques