Accurate Assessment via Process Data

TL;DR

This paper demonstrates that incorporating detailed process data from computer-based assessments significantly enhances the accuracy and reliability of student ability evaluations, both theoretically and empirically.

Contribution

It introduces a method to effectively include process data in assessments, improving precision over traditional response-based methods.

Findings

Including process data increases assessment reliability.

Theoretical models confirm the benefit of process data.

Empirical results show improved out-of-sample accuracy.

Abstract

Accurate assessment of students' ability is the key task of a test. Assessments based on final responses are the standard. As the infrastructure advances, substantially more information is observed. One of such instances is the process data that is collected by computer-based interactive items, which contain a student's detailed interactive processes. In this paper, we show both theoretically and empirically that appropriately including such information in the assessment will substantially improve relevant assessment precision. The precision is measured empirically by out-of-sample test reliability.